Repository: provectus/swiss-army-kube Branch: master Commit: ae10efa45a12 Files: 86 Total size: 189.6 KB Directory structure: gitextract_prbnparz/ ├── .github/ │ ├── ISSUE_TEMPLATE/ │ │ ├── bug_report.md │ │ └── enhancement.md │ ├── keylabeler.yml │ ├── prlint.json │ ├── release-drafter.yml │ ├── stale.yml │ └── workflows/ │ ├── reviewdog.yml │ └── tflint.yml ├── .gitignore ├── .pre-commit-config.yaml ├── .scripts/ │ ├── requirements.txt │ └── validate_variables.py ├── CONTRIBUTING.md ├── LICENSE.md ├── QUICKSTART.md ├── README.md ├── charts/ │ ├── README.md │ ├── aws-fsx-csi-driver/ │ │ ├── .helmignore │ │ ├── Chart.yaml │ │ ├── templates/ │ │ │ ├── cluster-role-binding.yaml │ │ │ ├── cluster-role.yaml │ │ │ ├── csi-driver.yaml │ │ │ ├── daemon-set.yaml │ │ │ ├── deployment.yaml │ │ │ └── service-account.yaml │ │ └── values.yaml │ └── cluster-issuers/ │ ├── .helmignore │ ├── Chart.yaml │ ├── README.md │ ├── templates/ │ │ └── clusterissuer.yaml │ └── values.yaml ├── docs/ │ ├── CONTRIBUTE_PR.md │ └── TROUBLESHOOTING.md ├── examples/ │ ├── README.md │ ├── argocd/ │ │ ├── .gitignore │ │ ├── README.md │ │ ├── locals.tf │ │ ├── main.tf │ │ ├── outputs.tf │ │ ├── providers.tf │ │ ├── variables.tf │ │ └── versions.tf │ ├── argocd-with-applications/ │ │ ├── .rspec │ │ ├── .ruby-version │ │ ├── Gemfile │ │ ├── README.md │ │ ├── Rakefile │ │ ├── main.tf │ │ ├── providers.tf │ │ ├── spec/ │ │ │ ├── .gitignore │ │ │ ├── acm_spec.rb │ │ │ ├── eks_spec.rb │ │ │ ├── route53_spec.rb │ │ │ └── spec_helper.rb │ │ ├── variables.tf │ │ └── versions.tf │ ├── common/ │ │ ├── CONFIGURE.md │ │ ├── README.md │ │ ├── custom.tf │ │ ├── destroy.sh │ │ ├── destroy_fix.sh │ │ ├── main.tf │ │ ├── outputs.tf │ │ ├── providers.tf │ │ ├── variables.tf │ │ └── versions.tf │ ├── docker-reverse-proxy/ │ │ ├── .rspec │ │ ├── .ruby-version │ │ ├── Gemfile │ │ ├── README.md │ │ ├── Rakefile │ │ ├── apps/ │ │ │ └── anchor │ │ ├── main.tf │ │ ├── providers.tf │ │ ├── spec/ │ │ │ ├── .gitignore │ │ │ ├── acm_spec.rb │ │ │ ├── eks_spec.rb │ │ │ ├── route53_spec.rb │ │ │ └── spec_helper.rb │ │ ├── variables.tf │ │ └── versions.tf │ └── test/ │ └── terraform_argocd_test_basic.go ├── go.mod ├── go.sum ├── modules/ │ └── README.md └── prerequisites_install.sh ================================================ FILE CONTENTS ================================================ ================================================ FILE: .github/ISSUE_TEMPLATE/bug_report.md ================================================ --- name: Bug report about: Create a report to help us improve title: '' labels: '' assignees: '' --- **Describe the bug** A clear and concise description of what the bug is. **Use version** What version you use **To Reproduce** Steps to reproduce the behavior: 1. Go to '...' 2. Click on '....' 3. Scroll down to '....' 4. See an error **Expected behavior** A clear and concise description of what you expected to happen. **Screenshots** If applicable, add screenshots to help explain your problem. **Additional context** Add any other context about the problem here. ================================================ FILE: .github/ISSUE_TEMPLATE/enhancement.md ================================================ --- name: Enhancement Request about: Suggest an enhancement to the SAK project labels: enhancemet --- **What would you like to be added**: **Why is this needed**:`. ================================================ FILE: .github/keylabeler.yml ================================================ # Determines if we search the title (optional). Defaults to true. matchTitle: true # Determines if we search the body (optional). Defaults to true. matchBody: true # Determines if label matching is case sensitive (optional). Defaults to true. caseSensitive: false # Explicit keyword mappings to labels. Form of match:label. Required. labelMappings: "[WIP]": WIP cicd: cicd issue: issue bug: BUG error: BUG help: help-wanted guidance: help-wanted feature: enhancement enhancement: enhancement feat: enhancement docs: docs chore: skip-changelog skip: skip-changelog argocd: sak-argocd argo-cd: sak-argocd albcontroller: sak-alb alb-controller: sak-alb cognito: sak-cognito certificate: sak-cert-manager letsencrypt: sak-cert-manager efk: sak-efk kibana: sak-efk elastic: sak-efk external-dns: sak-external-dns externaldns: sak-external-dns kubeflow: sak-kubeflow nginx: sak-nginx ingress: sak-nginx monitoring: sak-prometheus prometheus: sak-prometheus asg: sak-scaling scaling: sak-scaling oauth: sak-oauth ================================================ FILE: .github/prlint.json ================================================ { "title": [ { "pattern": "^(feature|fix|issue|bug|docs|cicd|refactor|test|Bump):\\s", "message": "Your title needs to be prefixed with a topic. Allowed prefixes: feature,fix,issue,bug,docs,cicd,refactor,test" } ], "body": [ { "pattern": ".{1,}", "message": "You need literally anything in your description" } ] } ================================================ FILE: .github/release-drafter.yml ================================================ name-template: 'v$RESOLVED_VERSION' tag-template: 'v$RESOLVED_VERSION' template: | ## Changes $CHANGES ## Contributors $CONTRIBUTORS exclude-labels: - 'skip-changelog' categories: - title: '🚀 Features' labels: - 'feature' - 'enhancement' - title: '🐛 Bug Fixes' labels: - 'fix' - 'bugfix' - 'bug' - title: '🧰 Maintenance' label: 'chore' change-template: '- $TITLE @$AUTHOR (#$NUMBER)' version-resolver: major: labels: - 'major' minor: labels: - 'minor' patch: labels: - 'patch' default: patch autolabeler: - label: 'chore' files: - '*.md' branch: - '/docs{0,1}\/.+/' - '/gh-pages\/.+/' - label: 'bug' branch: - '/fix\/.+/' - 'issues\/.+/' title: - '/fix/i' - 'issues/i' - label: 'enhancement' branch: - '/feature\/.+/' - '/new\/.+/' ================================================ FILE: .github/stale.yml ================================================ # Configuration for probot-stale - https://github.com/probot/stale # Number of days of inactivity before an Issue or Pull Request becomes stale daysUntilStale: 30 # Number of days of inactivity before an Issue or Pull Request with the stale label is closed. # Set to false to disable. If disabled, issues still need to be closed manually, but will remain marked as stale. daysUntilClose: 7 # Only issues or pull requests with all of these labels are check if stale. Defaults to `[]` (disabled) onlyLabels: [] # Issues or Pull Requests with these labels will never be considered stale. Set to `[]` to disable exemptLabels: - pinned - security - good first issue # Set to true to ignore issues in a project (defaults to false) exemptProjects: false # Set to true to ignore issues in a milestone (defaults to false) exemptMilestones: false # Set to true to ignore issues with an assignee (defaults to false) exemptAssignees: false # Label to use when marking as stale staleLabel: wontfix # Comment to post when marking as stale. Set to `false` to disable markComment: > This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. # Comment to post when removing the stale label. # unmarkComment: > # Your comment here. # Comment to post when closing a stale Issue or Pull Request. # closeComment: > # Your comment here. # Limit the number of actions per hour, from 1-30. Default is 30 limitPerRun: 30 # Limit to only `issues` or `pulls` only: pulls # Optionally, specify configuration settings that are specific to just 'issues' or 'pulls': pulls: daysUntilStale: 30 markComment: > This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. issues: exemptLabels: - confirmed - pinned - good first issue ================================================ FILE: .github/workflows/reviewdog.yml ================================================ name: reviewdog on: pull_request: paths: - '**.tf' - '**.tfvars' jobs: terraformPlanArgocd: env: AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} ZONE_ID: ${{ secrets.ZONE_ID }} name: runner / argocd runs-on: ubuntu-latest needs: tflint steps: - name: Clone repo uses: actions/checkout@master - name: Install Terraform run: | brew install tfenv tfenv install ${TERRAFORM_VERSION} - name: Terraform plan argocd working-directory: ./examples/argocd run: | tfenv use ${TERRAFORM_VERSION} terraform init --upgrade terraform plan -var 'cluster_name=swiss-army-github-ci' terraformPlanArgocdWithApplications: env: AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} ZONE_ID: ${{ secrets.ZONE_ID }} name: runner / argocd-with-applications runs-on: ubuntu-latest needs: tflint steps: - name: Clone repo uses: actions/checkout@master - name: Install Terraform run: | brew install tfenv tfenv install ${TERRAFORM_VERSION} - name: Terraform plan argocd-with-applications working-directory: ./examples/argocd-with-applications run: | tfenv use ${TERRAFORM_VERSION} terraform init --upgrade terraform plan -var "zone_id=${ZONE_ID}" -var 'cluster_name=swiss-army-github-ci' terraformPlanCommon: env: AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} ZONE_ID: ${{ secrets.ZONE_ID }} name: runner / common runs-on: ubuntu-latest needs: tflint steps: - name: Clone repo uses: actions/checkout@master - name: Install Terraform run: | brew install tfenv tfenv install ${TERRAFORM_VERSION} - name: Terraform plan common example working-directory: ./examples/common run: | tfenv use ${TERRAFORM_VERSION} terraform init --upgrade terraform plan -var "zone_id=${ZONE_ID}" -var 'cluster_name=swiss-army-github-ci' infracost: runs-on: ubuntu-latest needs: [terraformPlanCommon, terraformPlanArgocdWithApplications, terraformPlanArgocd] name: Show infracost diff steps: - name: Check out repository uses: actions/checkout@v2 - name: Run infracost diff common continue-on-error: true uses: infracost/infracost-gh-action@master # Use a specific version instead of master if locking is preferred env: INFRACOST_API_KEY: ${{ secrets.INFRACOST_API_KEY }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Do not change ZONE_ID: ${{ secrets.ZONE_ID }} # See the cloud credentials section for the options AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} with: entrypoint: /scripts/ci/diff.sh # Do not change path: ./examples/common terraform_plan_flags: -var "zone_id=${ZONE_ID}" -var 'cluster_name=swiss-army-github-ci' - name: Run infracost diff argocd continue-on-error: true uses: infracost/infracost-gh-action@master # Use a specific version instead of master if locking is preferred env: INFRACOST_API_KEY: ${{ secrets.INFRACOST_API_KEY }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Do not change # See the cloud credentials section for the options AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} with: entrypoint: /scripts/ci/diff.sh # Do not change path: ./examples/argocd terraform_plan_flags: -var 'cluster_name=swiss-army-github-ci' - name: Run infracost diff acgocd-with-applications continue-on-error: true uses: infracost/infracost-gh-action@master # Use a specific version instead of master if locking is preferred env: INFRACOST_API_KEY: ${{ secrets.INFRACOST_API_KEY }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Do not change ZONE_ID: ${{ secrets.ZONE_ID }} # See the cloud credentials section for the options AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} with: entrypoint: /scripts/ci/diff.sh # Do not change path: ./examples/argocd-with-applications terraform_plan_flags: -var "zone_id=${ZONE_ID}" -var 'cluster_name=swiss-army-github-ci' ================================================ FILE: .github/workflows/tflint.yml ================================================ name: tflint on: pull_request: paths: - '**.tf' - '**.tfvars' - '**.tfvars.json' - '**.hcl' jobs: tflint: name: runner / tflint runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 name: Checkout source code - uses: terraform-linters/setup-tflint@v2 name: Setup TFLint with: tflint_version: v0.41.0 github_token: ${{ secrets.github_token }} - name: Show version run: tflint --version - name: Init TFLint run: tflint --init - name: Run TFLint run: tflint -f compact ================================================ FILE: .gitignore ================================================ config-map-aws-auth_*.yaml kubeconfig_* .terraform local .idea *.tfstate* .cache kustomize kfctl.yaml *.hcl # use other *.tfvars files for parametrization of config-modules terraform.tfvars # direnv.net config .envrc *.tfvars ================================================ FILE: .pre-commit-config.yaml ================================================ repos: - repo: git://github.com/antonbabenko/pre-commit-terraform rev: v1.50.0 # Get the latest from: https://github.com/antonbabenko/pre-commit-terraform/releases hooks: - id: terraform_fmt # - id: terraform_docs # - id: terraform_validate # - id: terrascan # - id: checkov # - id: terraform_tflint ================================================ FILE: .scripts/requirements.txt ================================================ python-hcl2==2.0.0 ================================================ FILE: .scripts/validate_variables.py ================================================ #! /usr/bin/env python3 import hcl2 import glob import sys def get_variables(): variables = {} for file in glob.glob("**/variables.tf", recursive=True): with open(file, "r") as f: data = hcl2.load(f) for var in data['variable']: name = list(var.keys())[0] if name not in variables.keys(): variables[name] = {"definitions":[]} if "description" in list(var[name].keys()): v_description = var[name]["description"][0] else: v_description = "NULL" if "type" in list(var[name].keys()): v_type = var[name]["type"][0] else: v_type = "NULL" variables[name]["definitions"].append({"type": v_type, "description": v_description, "used_in": "/".join(file.split("/")[1:-1])}) return variables def validate(): result = 0 variables = get_variables() for var_name in variables.keys(): var = variables[var_name] problems = [] if len(set([i["description"] for i in var["definitions"]])) > 1: problems.append("description") if len(set([i["type"] for i in var["definitions"]])) > 1: problems.append("type") if len(problems) > 0: result = 1 sys.stderr.write("[\033[0;31mERROR\033[0m] Variable \033[0;32m%s\033[0m is used in %s, need to fix %s\n" % (var_name, ", ".join([i["used_in"] for i in var["definitions"]]), " and ".join(problems))) sys.exit(result) def check(name): v = get_variables()[name] sys.stdout.write("Variable \033[0;32m%s\033[0m:\n" % name) for d in v["definitions"]: sys.stdout.write("\tmodule: %s\n\ttype: %s\n\tdescription: %s\n\n" % (d['used_in'],d['type'],d['description'])) sys.stdout.write("Status: \033[0;32m%s\033[0m" % str(len(set([i["description"] for i in v["definitions"]])) == 1)) if __name__ == "__main__": if len(sys.argv) > 2: if sys.argv[1] == "check": check(sys.argv[2]) else: validate() ================================================ FILE: CONTRIBUTING.md ================================================ # Contributing Guide ## What You Can Help With Currently, we accept contributions to the documentation of modules, specifically README.md files of each module. You can contribute in the following ways: * Create a new Readme that is currently missing, add structure and write it in full or in part * Add missing sections, paragraphs, and information to existing READMEs * Correct typos and formatting issues * Add/propose graphics to illustrate documentation ## Getting Started Swiss Army Kube documentation welcomes improvements from all contributors, new and experienced. Anyone can contribute to the `github.com/provectus/swiss-army-kube` repository in the following formats: 1. Open an issue about the documentation 2. Propose a change with a pull request (PR) 3. Propose minor text/formatting edits right on GitHub without cloning All you need is being comfortable with Git and GitHub. ## Contributing Issues Find an [issue](https://github.com/provectus/swiss-army-kube/issues) to work on or create your own. If you are a new contributor take a look at issues marked with `good first issue`. ## Contributing Pull Requests (PRs) Please check our guide on how to contribute to Swiss Army Kube with PRs: * [Contributing Pull Requests](https://github.com/provectus/swiss-army-kube/blob/master/docs/CONTRIBUTE_PR.md) ## Select a Module to Contribute The list of all Swiss Army Kube modules below includes information on the current state of documentation for each Module. Please select a module to contribute: * [airflow](https://github.com/provectus/sak-incubator/tree/main/airflow) (has short annotation) * [cicd](https://github.com/provectus/sak-incubator/tree/main/cicd) (no Readme) + [argo](https://github.com/provectus/sak-incubator/tree/main/cicd/argo) (no Readme) + [jenkins](https://github.com/provectus/sak-incubator/tree/main/cicd/jenkins) (no Readme) * [ingress](https://github.com/provectus/sak-incubator/tree/main/ingress) (empty Readme) + [nginx](https://github.com/provectus/sak-incubator/tree/main/ingress/nginx) (has short annotation) + [alb-ingress](https://github.com/provectus/sak-incubator/tree/main/ingress/alb-ingress) (has short annotation) * [kubeflow](https://github.com/provectus/sak-incubator/tree/main/kubeflow) (has Readme) * **[kubernetes](https://github.com/provectus/sak-incubator/tree/main/kubernetes)** (has short annotation) * [logging](https://github.com/provectus/sak-incubator/tree/main/logging) (empty Readme) + [efk](https://github.com/provectus/sak-incubator/tree/main/logging/efk) (empty Readme) + [loki](https://github.com/provectus/sak-incubator/tree/main/logging/loki) (empty Readme) * [monitoring](https://github.com/provectus/sak-incubator/tree/main/monitoring) (empty Readme) + [prometheus](https://github.com/provectus/sak-incubator/tree/main/monitoring/prometheus) (no Readme) * **[network](https://github.com/provectus/sak-incubator/tree/main/network)** (empty Readme) * [rds](https://github.com/provectus/sak-incubator/tree/main/rds) (no Readme) * [scaling](https://github.com/provectus/sak-incubator/tree/main/scaling) (no Readme) * [storage](https://github.com/provectus/sak-incubator/tree/main/storage) (no Readme) + [efs](https://github.com/provectus/sak-incubator/tree/main/storage/efs) (empty Readme) + [fsx](https://github.com/provectus/sak-incubator/tree/main/storage/fsx) (no Readme) * **[system](https://github.com/provectus/sak-incubator/tree/main/system)** (has short annotation) ## Structure of a Module's README Approximate structure of a README.md document: * Module Description * Implementation * Usage * Configuration * Overrides In some cases, modules won't need some of these sections or require additional ones. Change it according to a particular module. Back to top ================================================ FILE: LICENSE.md ================================================ Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ================================================ FILE: QUICKSTART.md ================================================ # Quickstart ## Contents - [Prerequisites](#prerequisites) - [Usage](#usage) - [Repository Structure](#repostructure) - [Adding Developers to the Kubernetes Cluster](#adddevs) ## Prerequisites ### Setting up Amazon account and user First, you need to have an Amazon account and an IAM user with the "programmatic access" access type. * [Amazon AWS](https://aws.amazon.com/) Create an Amazon account, an IAM user, select your user, create access key on the Security Credentials tab, and make sure you saved your credentials. [Creating an IAM User in Your AWS Account](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html) Next, you have to install: * [Helm CLI](https://helm.sh/docs/intro/install/) * [Kubernetes CLI (kubectl)](https://kubernetes.io/docs/tasks/tools/install-kubectl/) * [Amazon CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html) * [AWS IAM Authenticator for Kubernetes](https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html) * [Terraform](https://learn.hashicorp.com/terraform/getting-started/install.html) * [Jq](https://stedolan.github.io/jq/) ### Installing Prerequisites on MacOS #### Automatic installation on MacOS For MacOS users this repository has the [prerequisites_install.sh](https://github.com/provectus/swiss-army-kube/blob/master/prerequisites_install.sh) script that will automatically install all the required prerequisites. Clone the repo and run the script: ``` bash swiss-army-kube/prerequisites_install.sh ``` There is no version of this script for users of other operating systems yet. Non-Mac users should check the link of each prerequisite and install everything manually following instructions. #### Manual installation on MacOS Alternatively, you can install the prerequisites manually one by one using Homebrew Cask. ``` $ brew install helm $ brew install kubernetes-cli $ brew install awscli $ brew install aws-iam-authenticator $ brew install terraform $ brew install jq ``` ### Installing Prerequisites on Linux An installation script for Linux users is to be done. At the moment use official guides linked above to install all prerequisites manually. You have to have them installed into the '/usr/local/bin' directory as a result. ## Usage ### 1. Clone this repository Clone this repo if you haven't done it yet: ``` git clone https://github.com/provectus/swiss-army-kube.git ``` ### 2. Go to the examples/xxx directory ``` cd swiss-army-kube/examples/common ``` The `examples/common` directory contains the common project structure. You can use this folder as is or rename it to your project/environment name for convenience. You can use any other example, more detailed instructions for each example can be found in the README of the example ### 3. Configure your EKS cluster Edit the `.tf` files to set cluster variables according to your project requirements. Check the [Configure Deployment](./examples/common/CONFIGURE.md) page to learn more. A `providers.tf` file contains configuration for Terraform providers, that define how to connect to each platform for working with it such as an AWS cloud or Kubernetes cluster, for example: to change AWS region from _us-west-2_ to another need to modify `region` option of provider `aws`. A `main.tf` files are consist of modules, each module provides infrastructure things. To add them you can uncomment modules block for specific services, some properties of modules are required, so please follow to module documentation under `modules` folder for additional information. ### 4. Deploy your pre-configured EKS cluster on Amazon with Terraform commands ``` terraform init terraform plan -out plan terraform apply "plan" ``` * `terraform init` initializes Terraform working directory. * `terraform plan` generates and shows an execution plan. * `terraform apply` builds infrastructure or applies changes to it. Check [Terraform CLI Commands](https://www.terraform.io/docs/commands/index.html) for more info. ### 5. Configure kubectl to manage your Kubernetes cluster kubectl is a CLI for Kubernetes cluster management. Make sure that the kubectl client version is within one minor version of your cluster's API server: ``` kubectl version --short ``` Kubectl uses a file named `config` to access Kubernetes clusters. Once you deployed a cluster with Terraform, the `config` file is automatically generated for the cluster in your project directory (`swiss-army-kube/examples/common`). By default, kubectl checks `$HOME/.kube` for the `config` file, so move it there. Alternatively, use an environment variable: ``` export KUBECONFIG=.config:$HOME/.kube/config ``` To view Kubernetes cluster details: ``` kubectl cluster-info ``` To view your current cluster configuration (shows merged kubeconfig settings or a specified kubeconfig file): ``` kubectl config view ``` Check that the deployment succeeded by listing currently deployed pods: ``` kubectl get pods ``` **6. Work with your EKS Kubernetes cluster.** #### Making changes to Deployment Apply changes (run after every change of your infrastructure code): ``` terraform plan -out plan terraform apply plan ``` #### Teardown and Cleanup To destroy any module: remove it from `main.tf` and run: ``` terraform plan -out plan && terraform apply plan ``` To destroy your EKS cluster: ``` terraform destroy ``` To destroy your EKS cluster with a script ignoring objects that can't be destroyed automatically without manual cleanup (recommended): ``` bash swiss-army-kube/examples/common/destroy.sh ``` It will ignore Route 53 zone resources, Amazon RDS for Kubeflow, argo-artifacts S3 bucket. Run the script, then destroy these objects manually one by one. ## Repository Structure The Swiss Army Kube repository has three main directories that provide a minimal set of resources allowing to comfortably start the development of a new IaC project: * `charts` - local Helm repository for Helm charts that can't be retrieved from public repositories. * `docs` - more detailed documentation and various FAQ's * `examples` - directories with different types of SAK use-cases to be used as a template for your projects that includes configuration files for modules and variables. * `modules` - Terraform modules (must-have and optional) to deploy your cluster with. ### Project Structure (Example Directory) The `swiss-army-kube/examples` directory contains project examples that you can use as boilerplates to start your new projects. Pick one, rename it to your project name for convenience, and modify the directory as required. This way you can create as many projects as you need really fast. To configure your project cluster for deployment, just [include modules](https://github.com/provectus/sak-incubator) that you need and [set variables](./examples/CONFIGURE.md) in the `.tf` files before deploying your EKS cluster with Terraform commands. The `examples/common` directory contains a set of `.tf` files: * `main.tf` - main file with infrastructure code * `providers.tf` - list of providers and their values The `examples/argocd-with-applications` the folder contains an example of deploying infrastructure in aws and applications for cluster operation (like external-dns, prometheus, cluster-autoscaler, etc.) The `examples/argocd` the folder contains an example of deploying infrastructure in aws and argo-cd server without any applications. Read about examples and how to use them in the example's README file. ## Adding Developers to Kubernetes Cluster ### DevOps engineer steps 1. Add an IAM user in AWS Console for developer with programmatic access 2. Add user ARN and name to `user_arns` variable with group `system:developers` for _kubernetes_ module in `main.tf` file 3. Run `terraform plan -out=plan` and review 4. Run `terraform apply plan` 5. Send `kubeconfig_internal-projects` config and IAM user tokens to the developer NOTE: To change developers' permissions on the Kubernetes cluster edit the `cluster_roles` variable in the `main.tf` file. ### Developer steps 1. Configure the AWS CLI with received tokens from DevOps engineer: ``` aws configure --profile your_profile_name ``` NOTE: To use your profile in console run: ``` export AWS_PROFILE=your_profile_name ``` 2. Use received `kubeconfig_internal-projects` config for running `kubectl` commands Command examples: ``` kubectl --kubeconfig=kubeconfig_internal-projects get pods --all-namespaces kubectl --kubeconfig=kubeconfig_internal-projects port-forward jenkins-xxxx-xxxx 5000:8080 -n jenkins ``` ================================================ FILE: README.md ================================================ [![Maintenance](https://img.shields.io/maintenance/yes/2023?style=for-the-badge)]() [![Apache2](https://img.shields.io/badge/license-Apache2-green.svg?style=for-the-badge)](https://www.apache.org/licenses/LICENSE-2.0) [![GitHub contributors](https://img.shields.io/github/contributors/provectus/swiss-army-kube?style=for-the-badge)](https://github.com/provectus/swiss-army-kube/graphs/contributors) ![GitHub release (latest SemVer)](https://img.shields.io/github/v/release/provectus/swiss-army-kube?style=for-the-badge) **[Quickstart](./QUICKSTART.md)** • **[Modules](./modules/README.md)** • **[Configure Deployment](./examples/common/CONFIGURE.md)** • **[Troubleshooting](./docs/TROUBLESHOOTING.md)** • **[Contributing](./CONTRIBUTING.md)** • **[Provectus](https://provectus.com/)** # Swiss Army Kube - Free IaC Tool for Easy EKS Kubernetes Cluster Deployment. logo  Swiss Army Kube (SAK) is an open-source IaC (Infrastructure as Code) collection of services for quick, easy, and controllable deployment of EKS Kubernetes clusters on Amazon for your projects. With Swiss Army Kube, cluster configuration and provisioning takes just a fraction of time normally spent on manual deployment via AWS management console. SAK automates deployments, making them repeatable, consistent, and less error-prone. Swiss Army Kube uses Terraform to describe the desired state of your infrastructure (resources that need to be provisioned like IAM roles, ASG, Route 53, subnets, etc.) and build a Kubernetes cluster on AWS EC2 instances. SAK provides example directories that you can use as easily modifiable templates to set up your cluster deployment configuration in minutes. All you need is to edit a couple of files to include modules and set variables. This way you can quickly configure and provision multiple dedicated EKS Kubernetes clusters with different configurations of modules, variables, networks, and Kubernetes versions. We believe that any developer or organization should be able to focus on their applications without having to worry too much about the nitty-gritty of infrastructure deployment. Currently, Swiss Army Kube is available for the [Amazon EKS](https://aws.amazon.com/eks/) (Elastic Kubernetes Service) for Kubernetes cluster only. We plan to expand to other platforms soon.
## Key Features ### Deploy * Provision an AWS EKS cluster in minutes * Use existing project structure to set up your infrastructure * Configure your deployment in a single `.tfvars` file * Add and configure modules in a single `.tf` file * Deploy with a couple of Terraform commands ### Manage * Manage your cluster with Terraform and Kubernetes CLI commands * Easily edit, reconfigure, rerun or destroy resources * Use handy scripts that make your work faster ### Scale * Configure and deploy as many projects as you need fast and easy * Scale deployments by adding new modules * Reduce your cloud infrastructure spend with spot instances * Maximize your workload cost-efficiency
## How it Works Configure and deploy as many projects as you want. 1. Sign up for Amazon account + Create and configure an IAM user 2. Install Prerequisites + Clone this repository + Install prerequisites via script (MacOS users) or manually (other users) 3. Configure your EKS cluster deployment using one of the `examples/` directories as a project template + Configure modules and variables 4. Deploy your EKS Kubernetes cluster with Terraform commands 5. Configure `kubectl` to manage your Kubernetes cluster 6. Manage your EKS Kubernetes cluster and deploy your containerized apps on it
## Get Started Visit our [Quickstart](./QUICKSTART.md) to install and configure prerequisites, set up your project deployment with desired modules and configurations in `*.tf` files, and deploy your infrastructure with Terraform commands: ``` terraform init terraform plan -out plan terraform apply "plan" ``` After deployment, manage your cluster with Terraform and Kubernetes CLI commands or AWS management console.
## Contributing Contributing to Swiss Army Kube is very welcome. Currently, we're looking for contributions to the documentation of [Modules](./modules). All you need is being comfortable with GitHub and Git. To get involved with documentation, please read our [Contributing Guide](./CONTRIBUTING.md).
## License Swiss Army Kube is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0.txt). ================================================ FILE: charts/README.md ================================================ # Charts ## Available charts The next charts are used for CI: - `cluster-issuers` - cert-manager crd. Get certificate from letsencrypt for ingress controller - `aws-fsx-csi-driver` - The Amazon FSx for Lustre Container Storage Interface (CSI) Driver implements CSI specification for container orchestrators (CO) to manage lifecycle of Amazon FSx for Lustre filesystems. ================================================ FILE: charts/aws-fsx-csi-driver/.helmignore ================================================ # Patterns to ignore when building packages. # This supports shell glob matching, relative path matching, and # negation (prefixed with !). Only one pattern per line. .DS_Store # Common VCS dirs .git/ .gitignore .bzr/ .bzrignore .hg/ .hgignore .svn/ # Common backup files *.swp *.bak *.tmp *~ # Various IDEs .project .idea/ *.tmproj .vscode/ ================================================ FILE: charts/aws-fsx-csi-driver/Chart.yaml ================================================ apiVersion: v2 name: aws-fsx-csi-driver description: A Helm chart for Kubernetes # A chart can be either an 'application' or a 'library' chart. # # Application charts are a collection of templates that can be packaged into versioned archives # to be deployed. # # Library charts provide useful utilities or functions for the chart developer. They're included as # a dependency of application charts to inject those utilities and functions into the rendering # pipeline. Library charts do not define any templates and therefore cannot be deployed. type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. version: 0.1.0 # This is the version number of the application being deployed. This version number should be # incremented each time you make changes to the application. appVersion: 1.16.0 ================================================ FILE: charts/aws-fsx-csi-driver/templates/cluster-role-binding.yaml ================================================ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: fsx-csi-external-provisioner-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: fsx-csi-external-provisioner-role subjects: - kind: ServiceAccount name: fsx-csi-controller-sa namespace: kube-system ================================================ FILE: charts/aws-fsx-csi-driver/templates/cluster-role.yaml ================================================ apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: fsx-csi-external-provisioner-role rules: - apiGroups: - "" resources: - persistentvolumes verbs: - get - list - watch - create - delete - apiGroups: - "" resources: - persistentvolumeclaims verbs: - get - list - watch - update - apiGroups: - storage.k8s.io resources: - storageclasses verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - list - watch - create - update - patch - apiGroups: - storage.k8s.io resources: - csinodes verbs: - get - list - watch - apiGroups: - "" resources: - nodes verbs: - get - list - watch - apiGroups: - coordination.k8s.io resources: - leases verbs: - get - watch - list - delete - update - create ================================================ FILE: charts/aws-fsx-csi-driver/templates/csi-driver.yaml ================================================ apiVersion: storage.k8s.io/v1beta1 kind: CSIDriver metadata: name: fsx.csi.aws.com namespace: kube-system spec: attachRequired: false ================================================ FILE: charts/aws-fsx-csi-driver/templates/daemon-set.yaml ================================================ apiVersion: apps/v1 kind: DaemonSet metadata: name: fsx-csi-node namespace: kube-system spec: selector: matchLabels: app: fsx-csi-node template: metadata: labels: app: fsx-csi-node spec: containers: - args: - --endpoint=$(CSI_ENDPOINT) - --logtostderr - --v=5 env: - name: CSI_ENDPOINT value: unix:/csi/csi.sock image: amazon/aws-fsx-csi-driver:v0.3.0 livenessProbe: failureThreshold: 5 httpGet: path: /healthz port: healthz initialDelaySeconds: 10 periodSeconds: 2 timeoutSeconds: 3 name: fsx-plugin ports: - containerPort: 9810 name: healthz protocol: TCP securityContext: privileged: true volumeMounts: - mountPath: /var/lib/kubelet mountPropagation: Bidirectional name: kubelet-dir - mountPath: /csi name: plugin-dir - args: - --csi-address=$(ADDRESS) - --kubelet-registration-path=$(DRIVER_REG_SOCK_PATH) - --v=5 env: - name: ADDRESS value: /csi/csi.sock - name: DRIVER_REG_SOCK_PATH value: /var/lib/kubelet/plugins/fsx.csi.aws.com/csi.sock - name: KUBE_NODE_NAME valueFrom: fieldRef: fieldPath: spec.nodeName image: quay.io/k8scsi/csi-node-driver-registrar:v1.1.0 name: csi-driver-registrar volumeMounts: - mountPath: /csi name: plugin-dir - mountPath: /registration name: registration-dir - args: - --csi-address=/csi/csi.sock - --health-port=9810 image: quay.io/k8scsi/livenessprobe:v1.1.0 imagePullPolicy: Always name: liveness-probe volumeMounts: - mountPath: /csi name: plugin-dir hostNetwork: true nodeSelector: kubernetes.io/arch: amd64 kubernetes.io/os: linux volumes: - hostPath: path: /var/lib/kubelet type: Directory name: kubelet-dir - hostPath: path: /var/lib/kubelet/plugins_registry/ type: Directory name: registration-dir - hostPath: path: /var/lib/kubelet/plugins/fsx.csi.aws.com/ type: DirectoryOrCreate name: plugin-dir ================================================ FILE: charts/aws-fsx-csi-driver/templates/deployment.yaml ================================================ apiVersion: apps/v1 kind: Deployment metadata: name: fsx-csi-controller namespace: kube-system spec: replicas: 2 selector: matchLabels: app: fsx-csi-controller template: metadata: labels: app: fsx-csi-controller spec: containers: - args: - --endpoint=$(CSI_ENDPOINT) - --logtostderr - --v=5 env: - name: CSI_ENDPOINT value: unix:///var/lib/csi/sockets/pluginproxy/csi.sock - name: AWS_ACCESS_KEY_ID valueFrom: secretKeyRef: key: key_id name: aws-secret optional: true - name: AWS_SECRET_ACCESS_KEY valueFrom: secretKeyRef: key: access_key name: aws-secret optional: true image: amazon/aws-fsx-csi-driver:v0.3.0 name: fsx-plugin volumeMounts: - mountPath: /var/lib/csi/sockets/pluginproxy/ name: socket-dir - args: - --timeout=5m - --csi-address=$(ADDRESS) - --v=5 - --enable-leader-election - --leader-election-type=leases env: - name: ADDRESS value: /var/lib/csi/sockets/pluginproxy/csi.sock image: quay.io/k8scsi/csi-provisioner:v1.3.0 name: csi-provisioner volumeMounts: - mountPath: /var/lib/csi/sockets/pluginproxy/ name: socket-dir nodeSelector: kubernetes.io/arch: amd64 kubernetes.io/os: linux priorityClassName: system-cluster-critical serviceAccount: fsx-csi-controller-sa tolerations: - key: CriticalAddonsOnly operator: Exists volumes: - emptyDir: {} name: socket-dir ================================================ FILE: charts/aws-fsx-csi-driver/templates/service-account.yaml ================================================ apiVersion: v1 kind: ServiceAccount metadata: name: fsx-csi-controller-sa namespace: kube-system {{- if .Values.serviceAccount.annotations }} annotations: {{- toYaml .Values.serviceAccount.annotations | nindent 4 }} {{- end }} ================================================ FILE: charts/aws-fsx-csi-driver/values.yaml ================================================ serviceAccount: annotations: {} ================================================ FILE: charts/cluster-issuers/.helmignore ================================================ # Patterns to ignore when building packages. # This supports shell glob matching, relative path matching, and # negation (prefixed with !). Only one pattern per line. .DS_Store # Common VCS dirs .git/ .gitignore .bzr/ .bzrignore .hg/ .hgignore .svn/ # Common backup files *.swp *.bak *.tmp *~ # Various IDEs .project .idea/ *.tmproj .vscode/ # products of "helm package" *.tgz ================================================ FILE: charts/cluster-issuers/Chart.yaml ================================================ apiVersion: v1 appVersion: "1.0" description: Letsencrypt certificate issuer name: cluster-issuers version: 0.1.0 ================================================ FILE: charts/cluster-issuers/README.md ================================================ # issuers This chart creates Letsencrypt certificate ClusterIssuer using DNS-01 challenge mechanism. More info about cert-manager processes: [cert-manager](https://docs.cert-manager.io/en/latest/tutorials/acme/quick-start/index.html#step-5-deploy-cert-manager) ## Prerequisites To use this chart you need helm chart for cert-manager installed in its own namespace: ``` helm install --name cert-manager \ --namespace cert-manager \ --version v0.11.0 \ jetstack/cert-manager ``` ## Configuration **FIXME**: change accessKeyID/secretAccessKey into using IAM policies Parameter | Description | Default --- | --- | --- | `accessKeyID` | | `` | | `secretAccessKey` | | `` | | `email` | E-mail to use in cert generation | `noreply@provectus.com` | | `region` | | `` | | `role` | IAM policies name | | `secretNamespace` | Namespace for secret holding `secretAccessKey` (optional) | `cert-manager` | | `hostedZoneID` | Route53 zone ID for managed zone (optional) | `` | ## Provided resources `ClusterIssuer` resources provided: * `letsencrypt-staging` - for use in development/testing to avoid bans and hitting rate limits. Is not trusted by SSL clients. * `letsencrypt-prod` - for production usage ## Example usage ```yaml --- apiVersion: cert-manager.io/v1alpha2 kind: Certificate metadata: name: mycert spec: secretName: mycert commonName: example.provectus.com dnsNames: - example.provectus.com issuerRef: name: letsencrypt-staging kind: ClusterIssuer ``` ================================================ FILE: charts/cluster-issuers/templates/clusterissuer.yaml ================================================ {{- $hostedZoneID := .Values.hostedZoneID | default "" -}} apiVersion: cert-manager.io/v1alpha2 kind: ClusterIssuer metadata: name: letsencrypt-staging spec: acme: server: https://acme-staging-v02.api.letsencrypt.org/directory email: {{ .Values.email }} privateKeySecretRef: name: letsencrypt-staging solvers: - dns01: route53: region: {{ .Values.region }} {{- if ne $hostedZoneID "" }} hostedZoneID: {{ .Values.hostedZoneID }} {{- end}} --- apiVersion: cert-manager.io/v1alpha2 kind: ClusterIssuer metadata: name: letsencrypt-prod spec: acme: server: https://acme-v02.api.letsencrypt.org/directory email: {{ .Values.email }} privateKeySecretRef: name: letsencrypt-prod solvers: - dns01: route53: region: {{ .Values.region }} {{- if ne $hostedZoneID "" }} hostedZoneID: {{ .Values.hostedZoneID }} {{- end}} ================================================ FILE: charts/cluster-issuers/values.yaml ================================================ # E-mail to use in cert generation email: "noreply@provectus.com" # AWS IAM role role: "" # AWS Region region: "" # (optional) Route53 zone ID for managed zone hostedZoneID: "" # (optional) Namespace for secret holding secretAccessKey secretNamespace: cert-manager ================================================ FILE: docs/CONTRIBUTE_PR.md ================================================ # Contributing Pull Requests This guide is written for contributing to documentation. It doesn't contain any instructions on installing software prerequisites. If your intended contribution requires any software installations, please refer to their respective official documentation. **Prerequisites** * Git installed on your local machine * GitHub account **Contents** 2. [PR Contribution Workflow](#workflow) 3. [Basic Workflow Example](#example) 4. [PR Acceptance policy](#accept) ## PR Contribution Workflow 1. [Fork and clone this repository (`git clone`)](#clonerepo) 2. [Create a feature branch against master (`git checkout -b featurename`)](#checkout) 3. [Make changes in the feature branch](#editdoc) 4. [Update the documentation](#docs) 5. [Use Linter to ensure correct syntax and formatting (`terraform fmt`, `pre-commit run -a`)](#lintit) 6. [Commit your changes (`git commit -am "Add a feature"`)](#commit) 7. [Push your changes to GitHub (`git push origin feature`)](#push) 8. [Open a Pull Request and wait for your PR to get reviewed](#openPR) 9. [Edit your PR to address feedback (if any)](#modifyPR) 10. [See your PR getting merged](#merged) ### 1. Fork and Clone this Repository In order to contribute, you need to make your own copy of the repository you're going to contribute to. You do this by forking the repository to your GitHub account and then cloning the fork to your local machine. 1. Fork this GitHub repository: on GitHub, navigate to the [main page of the repository](https://github.com/provectus/swiss-army-kube) and click the Fork button in the upper-right area of the screen. This will create a fork (a copy of this repository in your GitHub account). 2. Clone the fork and switch to the project directory by running in your terminal: ``` git clone https://github.com/provectus/swiss-army-kube.git cd swiss-army-kube ``` ### 2. Create a New Branch It is important to make all your changes in a separate branch created off the master branch. Before any modifications to the repository that you've just cloned, create a new branch off of the master branch. Create a new branch off of the current one and switch to it: ``` git checkout -b ``` To switch between branches, use the same command without the `-b` flag. For example, to switch back to the master branch: ``` git checkout master ``` This way you can switch between multiple branches when you work on multiple features at once. #### Branch Naming Conventions Give your branch a descriptive name so that others working on the project understand what you are working on. The branch name should include the name of the module that you're contributing to. Name your branch according to the following template, replacing `nginx` with the name of the module you're contributing to: ``` feature/docs_nginx ``` ### 3. Make Changes Make changes you want to propose. Make sure you do this in a dedicated branch based on the master branch. ### 4. Update the documentation Make sure to update the documentation. Each module should be properly documented: - the purpose of the module is stated; - pre-requisites and requirements for the module are given; - there is a list of input and output variables used inside the module. You can use [`terraform-docs`](https://github.com/terraform-docs/terraform-docs/) to automatically generate the documentation for the module ### 5. Use Linter for Correct Syntax & Formatting When applicable, use linters for Terraform to ensure proper formatting before committing and pushing your changes. Check your repository with one of them: * **[Terraform formatting](https://www.terraform.io/docs/commands/fmt.html)** - run `terraform fmt && tflint` * **[pre-commit-terraform](https://github.com/antonbabenko/pre-commit-terraform)** - run `pre-commit run -a` after installing and configuring the tool. ### 6. Commit Changes Commit changes often to avoid accidental data loss. Make sure to provide your commits with descriptive comments. ``` git add . git commit -m "Add description" ``` Or add and commit all changed files with one command: ``` git commit -am "Add description" ``` ### 7. Push Changes to GitHub Push your local changes to your fork on GitHub. ``` git push ``` For example, if your remote repository is called origin and you want to push a branch named Mod-argo: ``` git push origin Mod-argo ``` ### 8. Open a Pull Request Navigate to your fork on GitHub. Press the "New pull request" button in the upper-left part of the page. Add a title and a comment. Once you press the "Create pull request" button, the maintainers of this repository will receive your PR. ### 9. Address Feedback After you submit the PR, one or several of the Swiss Army Kube reviewers will provide you with actionable feedback. Edit your PR to address all of the comments. Reviewers do their best to provide feedback and approval in a timely fashion but note that response time may vary based on circumstances. ### 10. Your PR Gets Merged Once your PR is approved by a reviewer, it gets accepted and merged with the main repository. Merged PRs will get included in the next Swiss Army Kube release. ## Basic Workflow Example ``` git clone https://github.com/provectus/swiss-army-kube.git cd swiss-army-kube git checkout -b Mod-argo git status terraform fmt git commit -am "Add description" git push origin Mod-argo ``` ## PR Acceptance Policy What will make your PR more likely to get accepted: * Having your fixes on a dedicated branch * Proper branch naming * Descriptive commit messages * PR title describing what changed * PR comment describing why/where it changed in <80 chars * Texts checked for spelling and typos (you can use Grammarly) * Terraform code snippets checked with linters (when applicable) ### PR Title and Comment Conventions A PR title should describe what has changed. A PR comment should describe why and what/where. If your changes relate to a particular issue, a PR comment should contain an issue number. Please keep PR comments below 80 characters for readability. PR title rules (This rules will be checked by bot): 1. Must start with prefix. (Allowed prefixes: feature,fix,issue,bug,docs,cicd,refactor,test) 2. Description must not be empty. PR title example: ``` issue: added 2 sections and notification. System: new structure, description, minor fixes. ``` PR comment example: ``` Kubernetes: added sections: "Requirements", "How to update SAK module Kubeflow". Added version table showing the compatibility of Kubernetes and Kubeflow versions. Added notification about changes to what files will trigger Kubeflow terraform resources recreation. System: rearranged file structure according to best practice. Finished the module description section. Added a warning about using GPUs and a container. Fixed typos and formatting throughout the whole file. Issue #42 ``` Minor edits (typos, spelling, formatting, adding small text pieces) may get waved through. More substantial changes normally require more time, reviewers, and back-and-forths, and you might get asked for a PR resubmission or dividing changes into more that one PR. Usually, PRs are getting merged right after the approval. ================================================ FILE: docs/TROUBLESHOOTING.md ================================================ # Troubleshooting ### Debugging Terraform To set Terraform logs to the verbose mode: ``` export TF_LOG=trace ``` To remove corrupt state: ``` terraform state rm module.loki.helm_release.loki-stack ``` To refresh tfstate: ``` terraform refresh -var-file example.tfvars ``` To recreate resources: ``` terraform taint module.system.null_resource.helm_init ``` If `terraform destroy` command fails, run `destroy_fix.sh` and try `terraform destroy` again. After successful destroy process go to AWS console and delete argo-artifacts S3 bucket (if needed), also delete Route53 resources remaining from your deployment. Further reading: * [Debugging Terraform](https://www.terraform.io/docs/internals/debugging.html) ### Recreate IP address of NAT gateway (not seamlessly) Sometimes public IP address appears in blacklist of some services (e.g https://infra.apache.org/infra-ban.html). To avoid that - is good to prevent abuse such services using artifact caching servers such as Nexus, Artifactory, etc (if they have their own public IP) If there is still need to have direct access - it's time to just recreate IP. In `swiss-army-kube/example` (or `swiss-army-kube/`): ```shell terraform destroy -target 'module.network.module.vpc.aws_eip.nat[0]' terraform apply -target module.network.module.vpc ``` This method is suitable when short absence of access from VPC to internet is affordable (e.g during maintenance) ================================================ FILE: examples/README.md ================================================ # Examples of use SAK ## Known particular qualities of operation ### Usage of GPU nodes NVIDIA GPUs can be consumed via container level resource requirements using the resource name nvidia.com/gpu: ``` yaml apiVersion: v1 kind: Pod metadata: name: gpu-pod spec: containers: - name: cuda-container image: nvidia/cuda:9.0-devel resources: limits: nvidia.com/gpu: 2 # requesting 2 GPUs - name: digits-container image: nvidia/digits:6.0 resources: limits: nvidia.com/gpu: 2 # requesting 2 GPUs ``` __WARNING__: if you don't request GPUs when using the device plugin with NVIDIA images all the GPUs on the machine will be exposed inside your container. ### Hanging of Kubernetes namespaces on the deletion For some reason (unmanaged K8s resources, a large set of resources, etc), deletion of the Kubernetes namespace can take a while. In case of entire cluster destroying you could resolve this by manual deletion of resources from Terraform state, for example: ``` bash terraform state list | grep kubernetes_namespace | xargs terraform state rm {} ``` ================================================ FILE: examples/argocd/.gitignore ================================================ # General .DS_Store .AppleDouble .LSOverride .vscode/* !.vscode/settings.json !.vscode/tasks.json !.vscode/launch.json !.vscode/extensions.json !.vscode/*.code-snippets # Local History for Visual Studio Code .history/ # Built Visual Studio Code Extensions *.vsix ================================================ FILE: examples/argocd/README.md ================================================ # About That example demonstrates how to configure the EKS cluster with the ArgoCD application. A general idea of the usage of ArgoCD is managing all Kubernetes resources with it. ArgoCD provides us with a way of implementing the GitOps methodology for Kubernetes applications. ## Used modules - terraform-aws-modules/vpc/aws - terraform-aws-modules/eks/aws - github.com/provectus/sak-argocd Does not work with k8s with version 1.22, need to update helm chart ## Implementation First of all, you execute Terraform commands as it were for `common` example (please follow these instructions to understand how to use SAK). At this step, you will generate all required AWS resources such as EC2 instances, EKS cluster, IAM roles, etc. Also, Terraform will generate a few local files with ArgoCD applications. The next phase is it uploading these files to your GitHub repository. Please follow ArgoProj's documentation for more detailed information about [how it works](https://argoproj.github.io/argo-cd/#how-it-works) ## How to use That example creates a minimal EKS cluster without any additional software except ArgoCD. You can get KubeConfig for the newly created EKS cluster with the following aws-cli command: So for access, it needs to establish port forwarding for Kubernetes service, you can do it by the next command: ``` bash kubectl -n argocd port-forward svc/argocd-server 8080:80 ``` Now you can open in a browser, the password for accessing ArgoCD UI is stored in AWS System Manager Parameter store, you can retrieve it by command: ``` bash aws --region ssm get-parameter --with-decryption --name //argocd/password | jq -r '.Parameter.Value' ``` Login username is `admin`. ## NodeGroup types and basic examples Following examples will help you to set needed configuration for your environment: ### [General purpose instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/general-purpose-instances.html) General purpose instances provide a balance of compute, memory, and networking resources, and can be used for a wide range of workloads. Example ```hcl general = { # expected length of name to be in the range (1 - 38) name = "${local.environment}-${local.cluster_name}" max_size = 3 desired_size = 1 bootstrap_extra_args = "${local.default_bootstrap_extra_args} --kubelet-extra-args \"--node-labels=node-type=general,node.kubernetes.io/lifecycle=`curl -s http://169.254.169.254/latest/meta-data/instance-life-cycle`\"" use_mixed_instances_policy = true mixed_instances_policy = { instances_distribution = { on_demand_base_capacity = 0 on_demand_percentage_above_base_capacity = 10 spot_allocation_strategy = "capacity-optimized" } override = [ { instance_type = "m5.large" weighted_capacity = "1" }, { instance_type = "m6i.large" weighted_capacity = "2" }, ] } } ### [Compute optimized instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/compute-optimized-instances.html) Compute optimized instances are ideal for compute-bound applications that benefit from high-performance processors. Example ```hcl cpu-optimized = { # expected length of name to be in the range (1 - 38) name = "${local.environment}-${local.cluster_name}-cpu" max_size = 3 desired_size = 1 bootstrap_extra_args = "${local.default_bootstrap_extra_args} --kubelet-extra-args \"--node-labels=node-type=cpu-optimized,node.kubernetes.io/lifecycle=`curl -s http://169.254.169.254/latest/meta-data/instance-life-cycle`\"" use_mixed_instances_policy = true mixed_instances_policy = { instances_distribution = { on_demand_base_capacity = 0 on_demand_percentage_above_base_capacity = 10 spot_allocation_strategy = "capacity-optimized" } override = [ { instance_type = "c5.large" weighted_capacity = "1" }, { instance_type = "c6i.large" weighted_capacity = "2" }, ] } } ``` ### [Memory optimized instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/memory-optimized-instances.html) Memory optimized instances are designed to deliver fast performance for workloads that process large data sets in memory. Example ```hcl memory-optimized = { # expected length of name to be in the range (1 - 38) name = "${local.environment}-${local.cluster_name}-memory" max_size = 3 desired_size = 1 bootstrap_extra_args = "${local.default_bootstrap_extra_args} --kubelet-extra-args \"--node-labels=node-type=memory-optimized,node.kubernetes.io/lifecycle=`curl -s http://169.254.169.254/latest/meta-data/instance-life-cycle`\"" use_mixed_instances_policy = true mixed_instances_policy = { instances_distribution = { on_demand_base_capacity = 0 on_demand_percentage_above_base_capacity = 10 spot_allocation_strategy = "capacity-optimized" } override = [ { instance_type = "r5.large" weighted_capacity = "1" }, { instance_type = "r6i.large" weighted_capacity = "2" }, ] } } ``` ### [Accelerated computing instances(GPU optimized)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/accelerated-computing-instances.html) If you require high processing capability, you'll benefit from using accelerated computing instances, which provide access to hardware-based compute accelerators such as Graphics Processing Units (GPUs), Field Programmable Gate Arrays (FPGAs), or AWS Inferentia. - [GPU optimized instance types](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/accelerated-computing-instances.html#gpu-instances) An instance with an attached NVIDIA GPU, such as a P3 or G4dn instance, must have the appropriate NVIDIA driver installed. Depending on the instance type, you can either download a public NVIDIA driver, download a driver from Amazon S3 that is available only to AWS customers, or use an AMI with the driver pre-installed. - [Available drivers by instance type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-nvidia-driver.html#:~:text=for%20video%20decoding-,Available%20drivers%20by%20instance%20type,-The%20following%20table) - [AMIs with the NVIDIA drivers installed](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-nvidia-driver.html#:~:text=Option%201%3A%20AMIs%20with%20the%20NVIDIA%20drivers%20installed) - [Public NVIDIA drivers](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-nvidia-driver.html#:~:text=Option%202%3A%20Public%20NVIDIA%20drivers) - [GRID drivers (G5, G4dn, and G3 instances)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-nvidia-driver.html#:~:text=Option%203%3A%20GRID%20drivers%20(G5%2C%20G4dn%2C%20and%20G3%20instances)) - [NVIDIA gaming drivers (G5 and G4dn instances)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-nvidia-driver.html#:~:text=Option%204%3A%20NVIDIA%20gaming%20drivers%20(G5%20and%20G4dn%20instances)) An instance with an attached AMD GPU, such as a G4ad instance, must have the appropriate AMD driver installed. Depending on your requirements, you can either use an AMI with the driver preinstalled or download a driver from Amazon S3. - [AMIs with the AMD driver installed](https://aws.amazon.com/marketplace/search/results?searchTerms=AMD+Radeon+Pro+Driver&CREATOR=e6a5002c-6dd0-4d1e-8196-0a1d1857229b&filters=CREATOR) - [AMD driver download and install](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-amd-driver.html#amd-radeon-pro-software-for-enterprise-driver:~:text=.-,AMD%20driver%20download,-If%20you%20aren%27t) Example ```hcl gpu-optimized = { # expected length of name to be in the range (1 - 38) name = "${local.environment}-${local.cluster_name}-gpu" max_size = 3 desired_size = 1 bootstrap_extra_args = "${local.default_bootstrap_extra_args} --kubelet-extra-args \"--node-labels=node-type=gpu-optimized,node.kubernetes.io/lifecycle=`curl -s http://169.254.169.254/latest/meta-data/instance-life-cycle`\"" ami_id = "ami-xxxxxxxxxxxxxxxx" ## Choose ami id with hardware required drivers from aws marketplace(https://aws.amazon.com/marketplace/search) use_mixed_instances_policy = true mixed_instances_policy = { instances_distribution = { on_demand_base_capacity = 0 on_demand_percentage_above_base_capacity = 10 spot_allocation_strategy = "capacity-optimized" } override = [ { instance_type = "g4dn.xlarge" weighted_capacity = "1" } ] } } ``` ## [Fargate Profiles](https://docs.aws.amazon.com/eks/latest/userguide/fargate.html) Fargate is a technology that provides on-demand, right-sized compute capacity for containers. With Fargate, you don't have to provision, configure, or scale groups of virtual machines on your own to run containers. You also don't need to choose server types, decide when to scale your node groups, or optimize cluster packing. Example ```hcl fargate_profiles = { fargate-profile-example = { name = "${local.environment}-${local.cluster_name}-fargate" selectors = [ { namespace = "fargate" labels = { Application = "fargate-example" } } ] } } ``` ================================================ FILE: examples/argocd/locals.tf ================================================ locals { ### VPC locals zones = coalescelist(var.availability_zones, data.aws_availability_zones.available.names) cidr = var.cidr != null ? var.cidr : "10.${var.network}.0.0/16" private = var.cidr != null ? [for i, z in local.zones : cidrsubnet(local.cidr, var.network_delimiter, i)] : [for i, _ in local.zones : "10.${var.network}.20${i}.0/24"] public = var.cidr != null ? [for i, z in local.zones : cidrsubnet(local.cidr, var.network_delimiter, pow(2, var.network_delimiter) - i)] : [for i, _ in local.zones : "10.${var.network}.${i}.0/24"] #EKS module local environment = var.environment project = var.project cluster_name = var.cluster_name domain = ["${local.cluster_name}.${var.domain_name}"] subnets = module.vpc.private_subnets registry = "https://registry.${local.domain[0]}" docker_config_json = jsonencode( { "\"registry-mirrors\"" = ["\"${local.registry}\""] }) default_bootstrap_extra_args = (var.container_runtime == "containerd") ? "--container-runtime containerd" : "--docker-config-json ${local.docker_config_json}" } ================================================ FILE: examples/argocd/main.tf ================================================ data "aws_eks_cluster" "cluster" { name = module.eks.cluster_id } data "aws_eks_cluster_auth" "cluster" { name = module.eks.cluster_id } data "aws_availability_zones" "available" {} module "vpc" { source = "terraform-aws-modules/vpc/aws" version = "v2.64.0" name = "${local.environment}-${local.cluster_name}" cidr = local.cidr azs = local.zones private_subnets = local.private public_subnets = local.public enable_nat_gateway = true single_nat_gateway = var.single_nat enable_dns_hostnames = true enable_dns_support = true public_subnet_tags = { Name = "${local.environment}-${local.cluster_name}-public" KubernetesCluster = local.cluster_name Environment = local.environment Project = local.project "kubernetes.io/role/elb" = "1" "kubernetes.io/cluster/${local.cluster_name}" = "owned" "kubernetes.io/cluster/${local.cluster_name}" = "shared" } private_subnet_tags = { Name = "${local.environment}-${local.cluster_name}-private" "kubernetes.io/role/elb-internal" = "1" "kubernetes.io/cluster/${local.cluster_name}" = "owned" } tags = { Name = "${local.environment}-${local.cluster_name}" Environment = local.environment Project = local.project Terraform = "true" } } module "eks" { source = "terraform-aws-modules/eks/aws" version = "18.30.2" cluster_version = var.cluster_version cluster_name = local.cluster_name prefix_separator = "" iam_role_name = local.cluster_name cluster_security_group_name = local.cluster_name cluster_security_group_description = "EKS cluster security group." subnet_ids = local.subnets vpc_id = module.vpc.vpc_id enable_irsa = false manage_aws_auth_configmap = true create_aws_auth_configmap = true # NOTE: # enable cloudwatch logging cluster_enabled_log_types = var.cloudwatch_logging_enabled ? var.cloudwatch_cluster_log_types : [] cloudwatch_log_group_retention_in_days = var.cloudwatch_logging_enabled ? var.cloudwatch_cluster_log_retention_days : 90 tags = { Environment = local.environment Project = local.project } self_managed_node_group_defaults = { update_launch_template_default_version = true iam_role_additional_policies = [ "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore", "arn:aws:iam::aws:policy/ElasticLoadBalancingFullAccess", "arn:aws:iam::aws:policy/AmazonRoute53FullAccess", "arn:aws:iam::aws:policy/AmazonRoute53AutoNamingFullAccess", "arn:aws:iam::aws:policy/AmazonElasticFileSystemFullAccess", "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryFullAccess", ] additional_userdata = "sudo yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm && sudo systemctl enable amazon-ssm-agent && sudo systemctl start amazon-ssm-agent" bootstrap_extra_args = local.default_bootstrap_extra_args metadata_options = { http_endpoint = "enabled" http_tokens = "optional" } } # Note: # If you add here worker groups with GPUs or some other custom resources make sure # to start the node in ASG manually once or cluster autoscaler doesn't find the resources. # # After that autoscaler is able to see the resources on that ASG. # self_managed_node_groups = { general = { # expected length of name to be in the range (1 - 38) name = "${local.environment}-${local.cluster_name}" max_size = 3 desired_size = 1 bootstrap_extra_args = "${local.default_bootstrap_extra_args} --kubelet-extra-args \"--node-labels=node-type=general,node.kubernetes.io/lifecycle=`curl -s http://169.254.169.254/latest/meta-data/instance-life-cycle`\"" use_mixed_instances_policy = true mixed_instances_policy = { instances_distribution = { on_demand_base_capacity = 0 on_demand_percentage_above_base_capacity = 10 spot_allocation_strategy = "capacity-optimized" } override = [ { instance_type = "m5.large" weighted_capacity = "1" }, { instance_type = "m6i.large" weighted_capacity = "2" }, ] } } # worker_group = concat(local.common, local.cpu, local.gpu) } } # OIDC cluster EKS settings resource "aws_iam_openid_connect_provider" "cluster" { client_id_list = ["sts.amazonaws.com"] thumbprint_list = ["9e99a48a9960b14926bb7f3b02e22da2b0ab7280"] url = module.eks.cluster_oidc_issuer_url } module "argocd" { depends_on = [module.vpc.vpc_id, module.eks.cluster_id, data.aws_eks_cluster.cluster] source = "github.com/provectus/sak-argocd" branch = var.argocd.branch owner = var.argocd.owner repository = var.argocd.repository cluster_name = module.eks.cluster_id path_prefix = "examples/argocd/" domains = local.domain ingress_annotations = { "nginx.ingress.kubernetes.io/ssl-redirect" = "false" } conf = { "server.service.type" = "ClusterIP" "server.ingress.paths[0]" = "/" } } ================================================ FILE: examples/argocd/outputs.tf ================================================ output "vpc_id" { value = module.vpc.vpc_id } output "private_subnets" { value = module.vpc.private_subnets } output "private_subnets_cidr_blocks" { value = module.vpc.private_subnets_cidr_blocks } output "vpc" { value = module.vpc } output "cluster_name" { value = module.eks.cluster_id description = "Name of eks cluster deploy" } output "cluster_oidc_url" { value = module.eks.cluster_oidc_issuer_url description = "Oidc issuer url for EKS cluster" } output "cluster_output" { value = { "cluster_oidc_issuer_url" = module.eks.cluster_oidc_issuer_url, "oidc_provider_arn" = aws_iam_openid_connect_provider.cluster.arn, "cluster_id" = module.eks.cluster_id } } output "this" { value = module.eks description = "TBD" } ================================================ FILE: examples/argocd/providers.tf ================================================ provider "aws" { region = var.region } provider "kubernetes" { host = data.aws_eks_cluster.cluster.endpoint cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority[0].data) token = data.aws_eks_cluster_auth.cluster.token } provider "helm" { kubernetes { host = data.aws_eks_cluster.cluster.endpoint cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority[0].data) token = data.aws_eks_cluster_auth.cluster.token } } ================================================ FILE: examples/argocd/variables.tf ================================================ variable "cidr" { type = string description = "TBD" default = null } variable "network_delimiter" { type = string description = "TBD" default = "8" } variable "network" { type = string description = "Number would be used to template CIDR 10.X.0.0/16." default = "10" } variable "single_nat" { type = bool description = "Use single Nat gateway or separeta for all AZ" default = true } variable "cluster_name" { default = "swiss-army-kube" type = string description = "A name of the Amazon EKS cluster" } variable "region" { default = "eu-north-1" type = string description = "Set default region" } variable "availability_zones" { default = ["eu-north-1a", "eu-north-1b"] type = list(any) description = "Availability zones for project" } variable "environment" { default = "dev" type = string description = "A value that will be used in annotations and tags to identify resources with the `Environment` key" } variable "project" { default = "SWISS" type = string description = "A value that will be used in annotations and tags to identify resources with the `Project` key" } variable "domain_name" { default = "swiss.sak.ninja" type = string description = "Default domain name" } variable "argocd" { default = { repository = "swiss-army-kube" branch = "master" owner = "provectus" } type = map(string) description = "A set of values for enabling deployment through ArgoCD" } variable "cluster_version" { type = string description = "EKS cluster version" default = "1.22" } variable "container_runtime" { type = string default = "docker" description = "Type of container runtime interface. Allowed values: docker/containerd" validation { condition = can(regex("^(docker|containerd)$", var.container_runtime)) error_message = "Must be docker or containerd." } } # enable control plane cloudwatch logging variable "cloudwatch_logging_enabled" { type = bool description = "Send EKS control plane logs to cloudwatch" default = false } variable "cloudwatch_cluster_log_types" { type = list(any) description = "log types that you want to send to cloudwatch" default = ["api", "audit", "authenticator", "controllerManager", "scheduler"] } variable "cloudwatch_cluster_log_retention_days" { type = number description = "logs retention period in days (1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653, 0). 0 means logs will never expire." default = 90 } ================================================ FILE: examples/argocd/versions.tf ================================================ terraform { required_providers { aws = { source = "hashicorp/aws" version = ">= 4.35.0" } external = { source = "hashicorp/external" version = "2.2.2" } helm = { source = "hashicorp/helm" version = "2.7.1" } kubernetes = { source = "hashicorp/kubernetes" version = "2.14.0" } local = { source = "hashicorp/local" version = "2.2.3" } null = { source = "hashicorp/null" version = "3.1.1" } random = { source = "hashicorp/random" version = "3.4.3" } } required_version = ">= 0.15" } ================================================ FILE: examples/argocd-with-applications/.rspec ================================================ --color --format documentation ================================================ FILE: examples/argocd-with-applications/.ruby-version ================================================ 2.5.1 ================================================ FILE: examples/argocd-with-applications/Gemfile ================================================ source 'https://rubygems.org' gem 'awspec' gem 'hcl-checker' ================================================ FILE: examples/argocd-with-applications/README.md ================================================ # About That example is the standard deployment of the SAK cluster with managing applications through ArgoCD. The next modules are supported: - [ArgoCD](https://github.com/provectus/sak-argocd) - [Scaling](https://github.com/provectus/sak-scaling) - [External DNS](https://github.com/provectus/sak-external-dns) - [External Secrets](https://github.com/provectus/sak-external-secrets) - [Prometheus monitoring](https://github.com/provectus/sak-prometheus) - [Nginx Ingress](https://github.com/provectus/sak-nginx) # How to use 1. Create user 2. Add variables to variables.tf. See example below: > :warning: Pay attention to argocd variables repository\branch\owner ``` variable "cluster_name" { default = "swiss-army-kube" } variable "region" { default = "eu-north-1" } variable "availability_zones" { default = ["eu-north-1a", "eu-north-1b"] } variable "zone_id" { default = "666" } variable "environment" { default = "dev" } variable "project" { default = "EDUCATION" } variable "domain_name" { default = "edu.provectus.io" } variable "argocd" { default = { repository = "swiss-army-kube" branch = "main" owner = "provectus" } } ``` 3. After completed changes, use terraform init --upgrade && terraform apply command. If deployment success a new folder will be created with name apps. You need to commit this folder to the Github repository (see variable "ArgoCD" repository\branch\owner) 4. ArgoCD sync all apps and deploy all manifest. Enjoy! # Test * Install rvm ``` curl -sSL https://get.rvm.io | bash -s stable ``` * Install ruby * Run `bundle install` * Run `terraform plan && terraform apply` * Run `AWS_PROFILE=YOUR_PROFILE rake spec` # Known bugs ### Kubernetes namespace termination stuck * Run kubectl proxy * Create json file tmp.json ``` { "apiVersion": "v1", "kind": "Namespace", "metadata": { "name": "STUCK_NAMESPACE" }, "spec": { "finalizers": [] } } ``` * Run `curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8001/api/v1/namespaces/STUCK_NAMESPACE/finalize` ### ACM Certificate in state `Pending` Wait... wait... wait... ================================================ FILE: examples/argocd-with-applications/Rakefile ================================================ require 'rspec/core/rake_task' require 'hcl/checker' RSpec::Core::RakeTask.new('spec') task :default => :spec ================================================ FILE: examples/argocd-with-applications/main.tf ================================================ data "aws_eks_cluster" "cluster" { name = module.kubernetes.cluster_name } data "aws_eks_cluster_auth" "cluster" { name = module.kubernetes.cluster_name } data "aws_route53_zone" "this" { # name = "edu.provectus.io." zone_id = var.zone_id private_zone = false } locals { environment = var.environment project = var.project cluster_name = var.cluster_name domain = ["${local.cluster_name}.${var.domain_name}"] tags = { environment = local.environment project = local.project } } module "network" { source = "github.com/provectus/sak-vpc" #By default ?ref=HEAD availability_zones = var.availability_zones environment = local.environment project = local.project cluster_name = local.cluster_name network = 10 } module "kubernetes" { depends_on = [module.network] source = "github.com/provectus/sak-kubernetes" environment = local.environment project = local.project availability_zones = var.availability_zones cluster_name = local.cluster_name domains = local.domain vpc_id = module.network.vpc_id subnets = module.network.private_subnets } module "argocd" { depends_on = [module.network.vpc_id, module.kubernetes.cluster_name, data.aws_eks_cluster.cluster, data.aws_eks_cluster_auth.cluster] source = "github.com/provectus/sak-argocd" branch = var.argocd.branch owner = var.argocd.owner repository = var.argocd.repository cluster_name = module.kubernetes.cluster_name path_prefix = "examples/argocd-with-applications/" domains = local.domain ingress_annotations = { "nginx.ingress.kubernetes.io/ssl-redirect" = "false" "kubernetes.io/ingress.class" = "nginx" } conf = { "server.service.type" = "ClusterIP" "server.ingress.paths[0]" = "/" } } #Apps module "external_dns" { depends_on = [module.argocd] source = "github.com/provectus/sak-external-dns" cluster_name = module.kubernetes.cluster_name argocd = module.argocd.state mainzoneid = data.aws_route53_zone.this.zone_id hostedzones = local.domain tags = local.tags } module "scaling" { depends_on = [module.argocd] source = "github.com/provectus/sak-scaling" cluster_name = module.kubernetes.cluster_name argocd = module.argocd.state } module "clusterwide" { depends_on = [module.argocd] source = "terraform-aws-modules/acm/aws" version = "~> v2.12" domain_name = "*.${local.domain[0]}" subject_alternative_names = [ local.domain[0] ] zone_id = module.external_dns.zone_id validate_certificate = true #Disable if used private DNS and validate it manually wait_for_validation = false tags = local.tags } module "nginx-ingress" { depends_on = [module.clusterwide] source = "github.com/provectus/sak-nginx" cluster_name = module.kubernetes.cluster_name argocd = module.argocd.state conf = { "controller.service.targetPorts.http" = "http" "controller.service.targetPorts.https" = "http" "controller.service.annotations.service\\.beta\\.kubernetes\\.io/aws-load-balancer-ssl-cert" = module.clusterwide.this_acm_certificate_arn "controller.service.annotations.service\\.beta\\.kubernetes\\.io/aws-load-balancer-backend-protocol" = "http" "controller.service.annotations.service\\.beta\\.kubernetes\\.io/aws-load-balancer-ssl-ports" = "https" } tags = local.tags } module "alb-ingress" { depends_on = [module.argocd] source = "github.com/provectus/sak-alb-controller" cluster_name = module.kubernetes.cluster_name vpc_id = module.network.vpc_id argocd = module.argocd.state } # module "prometheus" { # depends_on = [module.argocd] # source = "github.com/provectus/sak-prometheus" # cluster_name = module.kubernetes.cluster_name # argocd = module.argocd.state # domains = local.domain # } # module "victoriametrics" { # depends_on = [module.argocd] # source = "github.com/provectus/sak-victoria-metrics" # cluster_name = module.kubernetes.cluster_name # argocd = module.argocd.state # domains = local.domain # } # module "cognito" { # depends_on = [module.argocd, module.clusterwide] # source = "github.com/provectus/sak-cognito" # cluster_name = module.kubernetes.cluster_name # domain = "${local.cluster_name}.${var.domain_name}" # zone_id = var.zone_id # mfa_configuration = "OPTIONAL" # acm_arn = module.clusterwide.this_acm_certificate_arn # tags = local.tags # } # module "external_secrets" { # depends_on = [module.argocd] # source = "github.com/provectus/sak-external-secrets" # cluster_oidc_url = module.kubernetes.cluster_oidc_url # cluster_name = module.kubernetes.cluster_name # argocd = module.argocd.state # tags = local.tags # } ================================================ FILE: examples/argocd-with-applications/providers.tf ================================================ provider "aws" { region = var.region } provider "kubernetes" { host = data.aws_eks_cluster.cluster.endpoint cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data) token = data.aws_eks_cluster_auth.cluster.token } provider "helm" { kubernetes { host = data.aws_eks_cluster.cluster.endpoint cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data) token = data.aws_eks_cluster_auth.cluster.token } } ================================================ FILE: examples/argocd-with-applications/spec/.gitignore ================================================ secrets.yml ================================================ FILE: examples/argocd-with-applications/spec/acm_spec.rb ================================================ require 'spec_helper' include TF cluster = TF.parseVars['variable']['cluster_name']['default'] domain = TF.parseVars['variable']['domain_name']['default'] describe acm("*.#{cluster}.#{domain}") do it { should exist } it { should be_issued} end ================================================ FILE: examples/argocd-with-applications/spec/eks_spec.rb ================================================ require 'spec_helper' include TF cluster = TF.parseVars['variable']['cluster_name']['default'] describe eks(cluster) do it { should exist } it { should be_active } its(:version) { should eq '1.18' } end ================================================ FILE: examples/argocd-with-applications/spec/route53_spec.rb ================================================ require 'spec_helper' include TF cluster = TF.parseVars['variable']['cluster_name']['default'] domain = TF.parseVars['variable']['domain_name']['default'] describe route53_hosted_zone("#{cluster}.#{domain}.") do it { should exist } its(:resource_record_set_count) { should eq 3 } end ================================================ FILE: examples/argocd-with-applications/spec/spec_helper.rb ================================================ require 'awspec' require 'hcl/checker' #Awsecrets.load(secrets_path: File.expand_path('./secrets.yml', File.dirname(__FILE__))) module TF def parseVars file_data = File.read("variables.tf") hcl = HCL::Checker.parse(file_data) hcl end end ================================================ FILE: examples/argocd-with-applications/variables.tf ================================================ variable "cluster_name" { default = "swiss-army" type = string description = "A name of the Amazon EKS cluster" } variable "region" { default = "eu-central-1" type = string description = "Set default region" } variable "availability_zones" { default = ["eu-central-1a", "eu-central-1b"] type = list(any) description = "Availability zones for project, minimum 2" } variable "zone_id" { default = "" type = string description = "Default zone id for root domain" #like Z04917561CQAI9UAF27D6 } variable "environment" { default = "dev" type = string description = "A value that will be used in annotations and tags to identify resources with the `Environment` key" } variable "project" { default = "SWISSARMY" type = string description = "A value that will be used in annotations and tags to identify resources with the `Project` key" } variable "domain_name" { default = "sak.ninja" type = string description = "Default domain name" } #Argocd sync repository variable "argocd" { default = { repository = "swiss-army-kube" branch = "master" owner = "provectus" } type = map(string) description = "A set of values for enabling deployment through ArgoCD" } ================================================ FILE: examples/argocd-with-applications/versions.tf ================================================ terraform { required_providers { aws = { source = "hashicorp/aws" version = ">= 3.56.0" } external = { source = "hashicorp/external" version = "2.1.0" } helm = { source = "hashicorp/helm" version = "2.1.2" } kubernetes = { source = "hashicorp/kubernetes" version = "2.2.0" } local = { source = "hashicorp/local" version = "2.1.0" } null = { source = "hashicorp/null" version = "3.1.0" } random = { source = "hashicorp/random" version = "3.1.0" } } required_version = ">= 0.15" } ================================================ FILE: examples/common/CONFIGURE.md ================================================ # Configure Deployment in example.tfvars ## Contents 1. [Common Variables](#Variables) 2. [Variables of Worker Nodes (Overview)](#nodevars) 3. [Variables of On-Demand Instances](#ondemvars) 4. [Using GPU Instances](#gpunodes) To configure your cluster with proper parameters before deployment, set variables for your project in the [`main.tf`](https://github.com/provectus/swiss-army-kube/blob/master/examples/common/main.tf) file. It contains variables for all modules in one place, making it easy to set up your configuration quickly and without having to work across multiple separate files for each module. ## Common Variables |Name |Description |Default / Example |-----------------------|-------------------------------|------------------------------- |`aws_region` | Name of your AWS region. Regions are physical locations where clusters of Amazon data centers are located. |`us-west-2` |`aws_private` | Deployment in either private (`true`) or public (`false`) mode.|`false` |`availability_zones` | Unique name of AWS cluster or several clusters. Each group of logical AWS data centers is called Availability Zone. | `"us-west-2b", "us-west-2a", "us-west-2c"` |`cluster_name` | Unique name of your Kubernetes cluster. | `yourclustername` |`environment` | Environment name tag for convenient search and identification of objects across your clusters. | `dev` |`project` | Project name tag for convenient search and identification of objects across your clusters. |`yourprojectname` |`mainzoneid` | Main Route 53 domain zone ID to let Terraform automatically add NS and SOA records to the root domain.|`Z02149423PVQ0YMP19F13` |`domains` | Your domain name or an array of domain names with the first being the main Ingress FQDN (Fully Qualified Domain Name) to create Route53 hosting zone and Kubernetes Ingress. |`swiss-army.edu.provectus.io` |`config_path` | Unique path to the Kubernetes configuration file for working with your EKS clusters. The configuration file is located in the `swiss-army-kube/example` directory. |`kubeconfig_projectname` |`network` | Set a subnet your cluster will work in by providing a number to be as used "x" in the 10.X.0.0/16 CIDR template. Normally the default is enough, but you can change it once you have complex installations that require other subnets. |`10` |`admin_arns` | Provide AWS IAM credentials (`userarn`, `username`, `groups`) of administrators of your Kubernetes cluster. Add as many administrators as you want by adding arrays.|`arn:aws:iam::245582572290:user/username`, `username`, `system:masters` |`cluster_version` | Provide a version of your EKS cluster. Swiss Army Kube supports EKS versions 1.14, 1.15, and 1.16. Deployments with Kubeflow require EKS 1.15. |`1.16` |`cert_manager_email` | Provide an email to let the cert-manager of your Kubernetes cluster sign up for free [Let’s Encrypt](https://letsencrypt.org/) certificates.|`youremail@domain.com` |`github-auth` | You can turn on Ingress Github Oauth 2 authorization to limit who can access your private services and validate users via Github. If `true`, provide `github-client-id`, `github-client-secret`, `cookie-secret`, `github-org`. |`false` |`google-auth ` | Set to `true` and fill if the block below if you want to use Ingress Google Auth.|`false` |`elasticDataSize` | If you use Kibana, provide PersistentVolume storage capacity here.|`30Gi` |`jenkins_password` | If you use Jenkins, provide a password here. Uncomment properties below to attach S3 read-only policy for Jenkins IAM roles or add needed policies. |`password` |`rds_database_name` | If you use RDS, provide a database name here.|`exampledb` |`rds_database_engine` | RDS database engine (aviable postgres mysql oracle-ee sqlserver-ex)| `postgres` |`rds_database_engine_version`| RDS databese engine version (see versions in AWS RDS engine table)|`9.6.9` |`rds_database_major_engine_version`| RDS databese major version (AWS bump minor version automatically)|`9` |`rds_database_instance`| Provide a RDS database instance type|`db.t3.large` |`rds_database_username`| Provide a RDS database username|`exampleuser` |`rds_database_password`| Provide a RDS database password| `""` |`rds_kms_key_id`| Provide a KMS key id if rds_storage_encrypted = true|`""` |`rds_allocated_storage`| Provide a RDS storage size in GB|`10` |`rds_storage_encrypted`| If you want encrypted database set true|`false` |`rds_maintenance_window`| Provide a maintenance window, at this time, the database may be unavailable (the window is set for updating)|`Mon:00:00-Mon:03:00` |`rds_backup_window`| Provide a backup window, at this time, the database may be unavailable (the window is set for creating backups)|`03:00-06:00` |`rds_database_multi_az`| If you want use multi aviability zone mode, set true.This will require an additional fee|`true` |`rds_database_delete_protection`| Delete protection mode, set true if you want prevent delete RDS|`false` |`rds_database_tags`| Additionals tags for RDS instance, comma separate key=value pairs|`{ "test" = "tags" }` |`airflow_username`| Provide a Airflow username here.|`username` |`airflow_password`| Provide a Airflow password here. If password null it's autogenerate and store to AWS ParamStore|`""` |`airflow_fernetKey`| Generate fernetKey (read about https://bcb.github.io/airflow/fernet-key )|`GFqrDfu-0oac6x2ATKLsx-Mr2yHKWFpa5hY4pYeWmXw=` |`airflow_postgresql_local`| Set true if you want use local postgresql database (pod in kubernetes).|`true` |`airflow_postgresql_host`| Provide postgresql host, if you set airflow_postgresql_local to false|`""` |`airflow_postgresql_port`| Provide postgresql port, if you set airflow_postgresql_local to false|`5432` |`airflow_postgresql_username`| Provide a postgresql username here.|`postgresqluser` |`airflow_postgresql_password`| Provide a postgresql password here.If password null it's autogenerate and store to AWS ParamStore|`""` |`airflow_postgresql_database`| Provide a postgresql database name here.|`airflow` |`airflow_redis_local`| Set true if you want use local redis database (pod in kubernetes).|`true` |`airflow_redis_host`| Provide redis host, if you set airflow_redis_local to false|`""` |`airflow_redis_port`| Provide redis port, if you set airflow_redis_local to false|`6379` |`airflow_redis_username`| Provide a redis username here.|`redisuser` |`airflow_redis_password`| Provide a redis password here.|`""` ## Variables of Worker Nodes (EC2 Instances) Use this block of variables to set up the type, number, and other parameters of your EC2 instances for the following node types: 1. Common - On-Demand EC2 instances 2. CPU - CPU-focused EC2 instances 3. GPU - GPU-focused EC2 instances **On-Demand (Common) Instances** With On-Demand instances, you pay for instances that you launch per hour or second, on a fixed-price basis. Unlike Spots, on-demand instances are stable and won't disappear out of the blue. We recommend, that at least 30% of your cluster consist of on-demand instances as the best practice. * [On-Demand Instances](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-on-demand-instances.html) * [Amazon EC2 On-Demand Pricing](https://aws.amazon.com/ec2/pricing/on-demand/) **Spot Instances** Spot instances are a spare capacity that AWS sells at up to 90% discount. Using them allows you to save money by optimizing workload costs. The downside is that Spots are not guaranteed to stay available and can be recalled by AWS at any time. The more spots you use in an EKS cluster, the more risk exists for your cluster to suddenly become unavailable. If you want use spot instance, set on_demand_common_percentage_above_base_capacity in percent. When set 30%, it's means that 70% instance in ASG common will be spot instance. **CPU Instances** Amazon EC2 C instances with high-performance processors. CPU instances are critical for resource-intensive tasks like scientific modeling, machine learning inference, and other compute-intensive apps. * [Amazon EC2 Compute Optimized Instances](https://aws.amazon.com/ec2/instance-types/#Compute_Optimized) **GPU Instances** Amazon EC2 P instances with high-performance processors. GPU instances use hardware accelerators, or co-processors, to perform functions more efficiently than is possible in software running on CPUs. GPUs are often used by data scientists to calculate ML models, perform functions, process graphics, etc. * [Amazon EC2 Accelerated Computing Instances](https://aws.amazon.com/ec2/instance-types/#Compute_Optimized) All nodes of a cluster are labeled with node type (Common, GPU, CPU). When you deploy an app, you can use Kubernetes tools to specify which group of nodes to deploy this particular app to. * [Check all Amazon EC2 Instance Types.](https://aws.amazon.com/ec2/instance-types/) ### Variables of On-Demand EC2 Instances |Name |Description |Default / Example |-----------------------|-------------------------------|------------------------------- |`on_demand_common_max_cluster_size` | Maximum number of nodes in a cluster |`5` |`on_demand_common_min_cluster_size ` | Minimum number of nodes in a cluster |`1` |`on_demand_common_desired_capacity` | How many nodes will get started |`2` |`on_demand_common_instance_type` | Amazon EC2 instance types in order of priority|`"m5.large", "m5.xlarge", "m5.2xlarge"` |`on_demand_common_allocation_strategy` | Allocation strategy that will define priority and number of different on-demand EC2 instance types in your cluster. Valid values: `prioritized`. |`prioritized` |`on_demand_common_base_capacity` | Percent of EC2 instances of this type in a cluster. Controls how much of the initial cluster capacity is made up of Common on-demand EC2 instances. Set to 0 indicates that you prefer to launch them as a percentage of the total group capacity that is running at any given time. |`0` |`on_demand_common_percentage_above_base_capacity ` | Controls the percentage of the add-on to the initial group that is made up of on-demand EC2 Instances versus the percentage that is made up of Spot Instances.|`100` |`on_demand_common_asg_recreate_on_change` | When true, recreates an ASG group if changes have been made. |`true` ## Using GPU Instances NVIDIA GPUs can now be consumed via container level resource requirements using the resource name `nvidia.com/gpu`: ``` apiVersion: v1 kind: Pod metadata: name: gpu-pod spec: containers: - name: cuda-container image: nvidia/cuda:9.0-devel resources: limits: nvidia.com/gpu: 2 # requesting 2 GPUs - name: digits-container image: nvidia/digits:6.0 resources: limits: nvidia.com/gpu: 2 # requesting 2 GPUs ``` WARNING: If you don't request GPUs when using the device plugin with NVIDIA images, all the GPUs on the machine will be exposed inside your container. ================================================ FILE: examples/common/README.md ================================================ # About simple installation of EKS and VPC # Prerequisites #### Helm v3 `brew install helm` #### kubectl `brew install kubernetes-cli` #### awscli `brew install awscli` #### aws-iam-authenticator `brew install aws-iam-authenticator` #### terraform `brew install terraform` #### kfctl `bash swiss-army-kube/kfctl_install.sh` ( To run kfctl, go to the `/usr/local/bin/kfctl` binary file in Finder, right-click, then select Open. Then click Open again to confirm that you want to open the app. ) #### jq `brew install jq` #### To install all prerequisites `bash swiss-army-kube/prerequisites_install.sh` # Structure main.tf - the main Terraform file with infrastructure code providers.tf - list of providers and their values # Deploy cluster Change variables.tf, choose modules in main.tf and do the following: Prepare and download modules `terraform init --upgrade=true` Plan and test deployment `terraform plan -out plan` Review plan if needed `terraform show plan` Deploy cluster and helm charts `terraform apply plan` ## Working with cluster To destroy some module just remove them from modules.tf and run `terraform plan -out plan && terraform apply plan` ## Troubleshooting Enable terraform logs verbose `export TF_LOG=trace` Remove corrupt state `terraform state rm module.loki.helm_release.loki-stack` Refresh tfstate `terraform refresh -var-file example.tfvars` Recreate resources `terraform taint module.system.null_resource.helm_init` If `terraform destroy` command fails, run `destroy_fix.sh` and try `terraform destroy` again. After successful destroy process go to AWS console and delete argo-artifacts S3 bucket (if needed), also delete Route53 resources remaining from your deployment. ================================================ FILE: examples/common/custom.tf ================================================ # module argocd { # source = "../../modules/cicd/argo-cd" # branch = var.branch # owner = var.owner # repository = var.repository # cluster_name = module.kubernetes.cluster_name # domains = var.domains # } # module cluster_autoscaler { # source = "../../modules/system/cluster-autoscaler" # image_tag = "v1.15.7" # cluster_name = module.kubernetes.cluster_name # module_depends_on = [module.kubernetes] # } # module cert_manager { # source = "../modules/system/cert-manager" # cluster_name = module.kubernetes.cluster_name # } # module external_secrets { # source = "../modules/system/external-secrets" # cluster_name = module.kubernetes.cluster_name # } # module external_dns { # source = "../../modules/system/external-dns" # cluster_name = module.kubernetes.cluster_name # environment = var.environment # project = var.project # vpc_id = module.network.vpc_id # aws_private = var.aws_private # domains = var.domains # mainzoneid = var.mainzoneid # } ================================================ FILE: examples/common/destroy.sh ================================================ #!/bin/bash SCRIPTPATH="$( cd "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )" terraform destroy -auto-approve RESULT=$? if [ $RESULT -ne 0 ]; then terraform state list | grep -e "module.*helm" | xargs terraform state rm terraform state list | grep -e "module.*route53" | xargs terraform state rm terraform state rm module.nginx.kubernetes_namespace.ingress-system || : terraform state rm module.system.kubernetes_namespace.cert-manager || : terraform state rm module.argo-artifacts.aws_s3_bucket.argo-artifacts || : terraform state rm module.nginx.kubernetes_secret.oauth2-proxy-secret-google || : terraform state rm module.nginx.kubernetes_secret.oauth2-proxy-secret || : case "$OSTYPE" in darwin*) sed -i "" 's/\"skip_final_snapshot\":.*/\"skip_final_snapshot\": true,/g' "$SCRIPTPATH/terraform.tfstate" ;; linux*) sed -i 's/\"skip_final_snapshot\":.*/\"skip_final_snapshot\": true,/g' "$SCRIPTPATH/terraform.tfstate" ;; esac terraform destroy -auto-approve fi ================================================ FILE: examples/common/destroy_fix.sh ================================================ #!/bin/bash SCRIPTPATH="$( cd "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )" # Workaround for failure of removing Route53 hosted zones terraform state list | grep -e "module.*helm" | xargs terraform state rm terraform state list | grep -e "module.*route53" | xargs terraform state rm terraform state rm module.nginx.kubernetes_namespace.ingress-system || : terraform state rm module.system.kubernetes_namespace.cert-manager || : # Workaround for non-empty s3 bucket terraform state rm module.argo-artifacts.aws_s3_bucket.argo-artifacts || : terraform state rm module.nginx.kubernetes_secret.oauth2-proxy-secret-google || : terraform state rm module.nginx.kubernetes_secret.oauth2-proxy-secret || : # Workaround for "Error: RDS Cluster FinalSnapshotIdentifier is required when a final snapshot is required" case "$OSTYPE" in darwin*) sed -i "" 's/\"skip_final_snapshot\":.*/\"skip_final_snapshot\": true,/g' "$SCRIPTPATH/terraform.tfstate" ;; linux*) sed -i 's/\"skip_final_snapshot\":.*/\"skip_final_snapshot\": true,/g' "$SCRIPTPATH/terraform.tfstate" ;; esac ================================================ FILE: examples/common/main.tf ================================================ data "aws_eks_cluster" "cluster" { name = module.kubernetes.cluster_name } data "aws_eks_cluster_auth" "cluster" { name = module.kubernetes.cluster_name } data "aws_route53_zone" "this" { # name = "edu.provectus.io." zone_id = var.zone_id private_zone = false } locals { environment = var.environment project = var.project cluster_name = var.cluster_name domain = ["${local.cluster_name}.${var.domain_name}"] tags = { environment = local.environment project = local.project } } module "network" { source = "github.com/provectus/sak-vpc" #By default ?ref=HEAD availability_zones = var.availability_zones environment = local.environment project = local.project cluster_name = local.cluster_name network = 10 } module "kubernetes" { depends_on = [module.network] source = "github.com/provectus/sak-kubernetes" environment = local.environment project = local.project availability_zones = var.availability_zones cluster_name = local.cluster_name domains = local.domain vpc_id = module.network.vpc_id subnets = module.network.private_subnets admin_arns = [ # { # userarn = "arn:aws:iam::xxxxxxxx:user/username" # username = "username" # groups = ["system:masters"] # } ] user_arns = [] } ================================================ FILE: examples/common/outputs.tf ================================================ ================================================ FILE: examples/common/providers.tf ================================================ provider "aws" { region = var.region } provider "kubernetes" { host = data.aws_eks_cluster.cluster.endpoint cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data) token = data.aws_eks_cluster_auth.cluster.token } provider "helm" { kubernetes { host = data.aws_eks_cluster.cluster.endpoint cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data) token = data.aws_eks_cluster_auth.cluster.token } } ================================================ FILE: examples/common/variables.tf ================================================ variable "cluster_name" { default = "swiss-army" type = string description = "A name of the Amazon EKS cluster" } variable "region" { default = "eu-central-1" type = string description = "Set default region" } variable "availability_zones" { default = ["eu-central-1a", "eu-central-1b"] type = list(any) description = "Availability zones for project, minimum 2" } variable "zone_id" { # default = " " #Comment for asking user after terraform apply type = string description = "Default zone id for root domain" #like Z04917561CQAI9UAF27D6 } variable "environment" { default = "dev" type = string description = "A value that will be used in annotations and tags to identify resources with the `Environment` key" } variable "project" { default = "EDUCATION" type = string description = "A value that will be used in annotations and tags to identify resources with the `Project` key" } variable "domain_name" { default = "example.com" type = string description = "Default domain name" } ================================================ FILE: examples/common/versions.tf ================================================ terraform { required_providers { aws = { source = "hashicorp/aws" version = ">= 3.56.0" } external = { source = "hashicorp/external" version = "2.1.0" } helm = { source = "hashicorp/helm" version = "2.1.2" } kubernetes = { source = "hashicorp/kubernetes" version = "2.2.0" } local = { source = "hashicorp/local" version = "2.1.0" } null = { source = "hashicorp/null" version = "3.1.0" } random = { source = "hashicorp/random" version = "3.1.0" } } required_version = ">= 0.15" } ================================================ FILE: examples/docker-reverse-proxy/.rspec ================================================ --color --format documentation ================================================ FILE: examples/docker-reverse-proxy/.ruby-version ================================================ 2.5.1 ================================================ FILE: examples/docker-reverse-proxy/Gemfile ================================================ source 'https://rubygems.org' gem 'awspec' gem 'hcl-checker' ================================================ FILE: examples/docker-reverse-proxy/README.md ================================================ # About That example is the standard deployment of the SAK cluster with managing applications through ArgoCD and internal registry with proxy mode enabled. It help to avoid docker-hub limits and reduce time to download docker images. The next modules are supported: - [ArgoCD](https://github.com/provectus/sak-argocd) - [Scaling](https://github.com/provectus/sak-scaling) - [External DNS](https://github.com/provectus/sak-external-dns) - [External Secrets](https://github.com/provectus/sak-external-secrets) - [Prometheus monitoring](https://github.com/provectus/sak-prometheus) - [Nginx Ingress](https://github.com/provectus/sak-nginx) - [Registry-mirror](https://github.com/provectus/sak-incubator/tree/main/registry-mirror) # How to use 1. Create user 2. Add variables to variables.tf. See example below: > :warning: Pay attention to argocd variables repository\branch\owner ``` variable "cluster_name" { default = "swiss-army-kube" } variable "region" { default = "eu-north-1" } variable "availability_zones" { default = ["eu-north-1a", "eu-north-1b"] } variable "zone_id" { default = "666" } variable "environment" { default = "dev" } variable "project" { default = "EDUCATION" } variable "domain_name" { default = "edu.provectus.io" } variable "argocd" { default = { repository = "swiss-army-kube" branch = "main" owner = "provectus" } } ``` 3. After completed changes, use terraform init --upgrade && terraform apply command. If deployment success a new folder will be created with name apps. You need to commit this folder to the Github repository (see variable "ArgoCD" repository\branch\owner) 4. ArgoCD sync all apps and deploy all manifest. Enjoy! # Test * Install rvm ``` curl -sSL https://get.rvm.io | bash -s stable ``` * Install ruby * Run `bundle install` * Run `terraform plan && terraform apply` * Run `AWS_PROFILE=YOUR_PROFILE rake spec` # Known bugs ### Kubernetes namespace termination stuck * Run kubectl proxy * Create json file tmp.json ``` { "apiVersion": "v1", "kind": "Namespace", "metadata": { "name": "STUCK_NAMESPACE" }, "spec": { "finalizers": [] } } ``` * Run `curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8001/api/v1/namespaces/STUCK_NAMESPACE/finalize` ### ACM Certificate in state `Pending` Wait... wait... wait... ================================================ FILE: examples/docker-reverse-proxy/Rakefile ================================================ require 'rspec/core/rake_task' require 'hcl/checker' RSpec::Core::RakeTask.new('spec') task :default => :spec ================================================ FILE: examples/docker-reverse-proxy/apps/anchor ================================================ ================================================ FILE: examples/docker-reverse-proxy/main.tf ================================================ data "aws_eks_cluster" "cluster" { name = module.kubernetes.cluster_name } data "aws_eks_cluster_auth" "cluster" { name = module.kubernetes.cluster_name } data "aws_route53_zone" "this" { # name = "edu.provectus.io." zone_id = var.zone_id private_zone = false } locals { environment = var.environment project = var.project cluster_name = var.cluster_name domain = ["${local.cluster_name}.${var.domain_name}"] tags = { environment = local.environment project = local.project } } module "network" { source = "github.com/provectus/sak-vpc" #By default ?ref=HEAD availability_zones = var.availability_zones environment = local.environment project = local.project cluster_name = local.cluster_name network = 10 } module "kubernetes" { depends_on = [module.network] source = "github.com/provectus/sak-kubernetes" environment = local.environment project = local.project availability_zones = var.availability_zones cluster_name = local.cluster_name domains = local.domain vpc_id = module.network.vpc_id subnets = module.network.private_subnets } module "argocd" { depends_on = [module.network.vpc_id, module.kubernetes.cluster_name, data.aws_eks_cluster.cluster, data.aws_eks_cluster_auth.cluster] source = "github.com/provectus/sak-argocd" branch = var.argocd.branch owner = var.argocd.owner repository = var.argocd.repository cluster_name = module.kubernetes.cluster_name path_prefix = "examples/docker-reverse-proxy/" domains = local.domain ingress_annotations = { "nginx.ingress.kubernetes.io/ssl-redirect" = "false" "kubernetes.io/ingress.class" = "nginx" } conf = { "server.service.type" = "ClusterIP" "server.ingress.paths[0]" = "/" } } #Apps module "external_dns" { depends_on = [module.argocd] source = "github.com/provectus/sak-external-dns" cluster_name = module.kubernetes.cluster_name argocd = module.argocd.state mainzoneid = data.aws_route53_zone.this.zone_id hostedzones = local.domain tags = local.tags } module "scaling" { depends_on = [module.argocd] source = "github.com/provectus/sak-scaling" cluster_name = module.kubernetes.cluster_name argocd = module.argocd.state } module "cert-manager" { depends_on = [module.argocd] source = "github.com/provectus/sak-cert-manager" cluster_name = module.kubernetes.cluster_name vpc_id = module.network.vpc_id argocd = module.argocd.state email = "dkharlamov@provectus.com" zone_id = module.external_dns.zone_id domains = local.domain } module "nginx-ingress" { depends_on = [module.argocd] source = "github.com/provectus/sak-nginx" cluster_name = module.kubernetes.cluster_name argocd = module.argocd.state conf = {} tags = local.tags } module "internal-nginx-ingress" { depends_on = [module.argocd] source = "github.com/provectus/sak-nginx" namespace_name = "internal-ingress" internal = true cluster_name = module.kubernetes.cluster_name argocd = module.argocd.state conf = { "controller.service.internal.enabled" = true "controller.service.annotations.service\\.beta\\.kubernetes\\.io/aws-load-balancer-internal" = "0.0.0.0" "controller.ingressClass" = "internal" } tags = local.tags } module "registry-mirror" { depends_on = [module.argocd] source = "github.com/provectus/sak-incubator//registry-mirror" cluster_name = module.kubernetes.cluster_name argocd = module.argocd.state storage = "filesystem" domains = local.domain conf = {} tags = local.tags } ================================================ FILE: examples/docker-reverse-proxy/providers.tf ================================================ provider "aws" { region = var.region } provider "kubernetes" { host = data.aws_eks_cluster.cluster.endpoint cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data) token = data.aws_eks_cluster_auth.cluster.token } provider "helm" { kubernetes { host = data.aws_eks_cluster.cluster.endpoint cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data) token = data.aws_eks_cluster_auth.cluster.token } } ================================================ FILE: examples/docker-reverse-proxy/spec/.gitignore ================================================ secrets.yml ================================================ FILE: examples/docker-reverse-proxy/spec/acm_spec.rb ================================================ require 'spec_helper' include TF cluster = TF.parseVars['variable']['cluster_name']['default'] domain = TF.parseVars['variable']['domain_name']['default'] describe acm("*.#{cluster}.#{domain}") do it { should exist } it { should be_issued} end ================================================ FILE: examples/docker-reverse-proxy/spec/eks_spec.rb ================================================ require 'spec_helper' include TF cluster = TF.parseVars['variable']['cluster_name']['default'] describe eks(cluster) do it { should exist } it { should be_active } its(:version) { should eq '1.18' } end ================================================ FILE: examples/docker-reverse-proxy/spec/route53_spec.rb ================================================ require 'spec_helper' include TF cluster = TF.parseVars['variable']['cluster_name']['default'] domain = TF.parseVars['variable']['domain_name']['default'] describe route53_hosted_zone("#{cluster}.#{domain}.") do it { should exist } its(:resource_record_set_count) { should eq 3 } end ================================================ FILE: examples/docker-reverse-proxy/spec/spec_helper.rb ================================================ require 'awspec' require 'hcl/checker' #Awsecrets.load(secrets_path: File.expand_path('./secrets.yml', File.dirname(__FILE__))) module TF def parseVars file_data = File.read("variables.tf") hcl = HCL::Checker.parse(file_data) hcl end end ================================================ FILE: examples/docker-reverse-proxy/variables.tf ================================================ variable "cluster_name" { default = "swiss-army" type = string description = "A name of the Amazon EKS cluster" } variable "region" { default = "eu-central-1" type = string description = "Set default region" } variable "availability_zones" { default = ["eu-central-1a", "eu-central-1b"] type = list(any) description = "Availability zones for project, minimum 2" } variable "zone_id" { #default = "" type = string description = "Default zone id for root domain" #like Z04917561CQAI9UAF27D6 } variable "environment" { default = "dev" type = string description = "A value that will be used in annotations and tags to identify resources with the `Environment` key" } variable "project" { default = "SWISSARMY" type = string description = "A value that will be used in annotations and tags to identify resources with the `Project` key" } variable "domain_name" { default = "sak.ninja" type = string description = "Default domain name" } #Argocd sync repository variable "argocd" { default = { repository = "swiss-army-kube" branch = "master" owner = "provectus" } type = map(string) description = "A set of values for enabling deployment through ArgoCD" } ================================================ FILE: examples/docker-reverse-proxy/versions.tf ================================================ terraform { required_providers { aws = { source = "hashicorp/aws" version = ">= 3.56.0" } external = { source = "hashicorp/external" version = "2.1.0" } helm = { source = "hashicorp/helm" version = "2.1.2" } kubernetes = { source = "hashicorp/kubernetes" version = "2.2.0" } local = { source = "hashicorp/local" version = "2.1.0" } null = { source = "hashicorp/null" version = "3.1.0" } random = { source = "hashicorp/random" version = "3.1.0" } } required_version = ">= 0.15" } ================================================ FILE: examples/test/terraform_argocd_test_basic.go ================================================ package test import ( "testing" "github.com/gruntwork-io/terratest/modules/terraform" "github.com/stretchr/testify/assert" ) func TestArgoCDApp(t *testing.T) { // Run this test in parallel with all the others t.Parallel() // Unique ID to namespace resources // uniqueId := random.UniqueId() // Generate a unique name for each VPC so tests running in parallel don't clash // vpcName := fmt.Sprintf("test-vpc-%s", uniqueId) // Generate a unique key in the S3 bucket for the Terraform state // backendS3Key := fmt.Sprintf("/%s/terraform.tfstate", uniqueId) terraformOptions := &terraform.Options{ // Where the Terraform code is located TerraformDir: "../argocd", // Variables to pass to the Terraform code Vars: map[string]interface{}{ "region": "eu-north-1", "cluster_name": "swiss-army-kube-sub2zero", // "argocd": { // "branch": "testlab", // "owner": "sub2zero", // } }, // Vars: map[string]interface{}{ // "aws_region": "us-east-2", // "aws_account_id": "111122223333", // ID of testing account // "vpc_name": vpcName, // "cidr_block": "10.0.0.0/16", // "num_nat_gateways": 1, // }, // Backend configuration to pass to the Terraform code // BackendConfig: map[string]interface{}{ // "bucket": "", // bucket in testing account // "region": "us-east-2", // region of bucket in testing account // "key": backendS3Key, // }, } // Run 'terraform destroy' at the end of the test to clean up defer terraform.Destroy(t, terraformOptions) // Run 'terraform init' and 'terraform apply' to deploy the module terraform.InitAndApply(t, terraformOptions) // Run `terraform output` to get the value of an output variable vpcCidr := terraform.Output(t, terraformOptions, "vpc_cidr") // Verify we're getting back the outputs we expect assert.Equal(t, "10.10.0.0/16", vpcCidr) // Run `terraform output` to get the value of an output variable privateSubnetCidrs := terraform.OutputList(t, terraformOptions, "private_subnets_cidr_blocks") // Verify we're getting back the outputs we expect assert.Equal(t, []string{"10.10.200.0/24", "10.10.201.0/24"}, privateSubnetCidrs) // Run `terraform output` to get the value of an output variable publicSubnetCidrs := terraform.OutputList(t, terraformOptions, "public_subnets_cidr_blocks") // Verify we're getting back the outputs we expect assert.Equal(t, []string{"10.10.0.0/24", "10.10.1.0/24"}, publicSubnetCidrs) // Run `terraform output` to get the value of an output variable eksClusterId := terraform.Output(t, terraformOptions, "cluster_name") // Verify we're getting back the outputs we expect assert.Equal(t, "swiss-army-kube-sub2zero", eksClusterId) } ================================================ FILE: go.mod ================================================ module swiss-army-knife go 1.19 require github.com/gruntwork-io/terratest v0.40.23 require ( cloud.google.com/go v0.83.0 // indirect cloud.google.com/go/storage v1.10.0 // indirect github.com/agext/levenshtein v1.2.3 // indirect github.com/apparentlymart/go-textseg/v13 v13.0.0 // indirect github.com/aws/aws-sdk-go v1.40.56 // indirect github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d // indirect github.com/davecgh/go-spew v1.1.1 // indirect github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e // indirect github.com/golang/protobuf v1.5.2 // indirect github.com/golang/snappy v0.0.3 // indirect github.com/googleapis/gax-go/v2 v2.0.5 // indirect github.com/hashicorp/errwrap v1.0.0 // indirect github.com/hashicorp/go-cleanhttp v0.5.2 // indirect github.com/hashicorp/go-getter v1.6.1 // indirect github.com/hashicorp/go-multierror v1.1.0 // indirect github.com/hashicorp/go-safetemp v1.0.0 // indirect github.com/hashicorp/go-version v1.3.0 // indirect github.com/hashicorp/hcl/v2 v2.9.1 // indirect github.com/hashicorp/terraform-json v0.13.0 // indirect github.com/jinzhu/copier v0.0.0-20190924061706-b57f9002281a // indirect github.com/jmespath/go-jmespath v0.4.0 // indirect github.com/jstemmer/go-junit-report v0.9.1 // indirect github.com/klauspost/compress v1.13.0 // indirect github.com/mattn/go-zglob v0.0.2-0.20190814121620-e3c945676326 // indirect github.com/mitchellh/go-homedir v1.1.0 // indirect github.com/mitchellh/go-testing-interface v1.0.0 // indirect github.com/mitchellh/go-wordwrap v1.0.1 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect github.com/stretchr/testify v1.7.0 // indirect github.com/tmccombs/hcl2json v0.3.3 // indirect github.com/ulikunitz/xz v0.5.8 // indirect github.com/zclconf/go-cty v1.9.1 // indirect go.opencensus.io v0.23.0 // indirect golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a // indirect golang.org/x/lint v0.0.0-20210508222113-6edffad5e616 // indirect golang.org/x/mod v0.4.2 // indirect golang.org/x/net v0.0.0-20210614182718-04defd469f4e // indirect golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c // indirect golang.org/x/sys v0.0.0-20220517195934-5e4e11fc645e // indirect golang.org/x/text v0.3.6 // indirect golang.org/x/tools v0.1.2 // indirect golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect google.golang.org/api v0.47.0 // indirect google.golang.org/appengine v1.6.7 // indirect google.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c // indirect google.golang.org/grpc v1.38.0 // indirect google.golang.org/protobuf v1.26.0 // indirect gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect ) ================================================ FILE: go.sum ================================================ cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU= cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU= cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY= cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc= cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0= cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To= cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4= cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M= cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc= cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk= cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs= cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc= cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY= cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI= cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk= cloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg= cloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8= cloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0= cloud.google.com/go v0.83.0 h1:bAMqZidYkmIsUqe6PtkEPT7Q+vfizScn+jfNA6jwK9c= cloud.google.com/go v0.83.0/go.mod h1:Z7MJUsANfY0pYPdw0lbnivPx4/vhy/e2FEkSkF7vAVY= cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg= cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc= cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ= cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU= cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw= cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos= cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk= cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs= cloud.google.com/go/storage v1.10.0 h1:STgFzyU5/8miMl0//zKh2aQeTyeaUH3WN9bSUiJ09bA= cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0= dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= github.com/agext/levenshtein v1.2.1/go.mod h1:JEDfjyjHDjOF/1e4FlBE/PkbqA9OfWu2ki2W0IB5558= github.com/agext/levenshtein v1.2.3 h1:YB2fHEn0UJagG8T1rrWknE3ZQzWM06O8AMAatNn7lmo= github.com/agext/levenshtein v1.2.3/go.mod h1:JEDfjyjHDjOF/1e4FlBE/PkbqA9OfWu2ki2W0IB5558= github.com/apparentlymart/go-dump v0.0.0-20180507223929-23540a00eaa3/go.mod h1:oL81AME2rN47vu18xqj1S1jPIPuN7afo62yKTNn3XMM= github.com/apparentlymart/go-textseg v1.0.0/go.mod h1:z96Txxhf3xSFMPmb5X/1W05FF/Nj9VFpLOpjS5yuumk= github.com/apparentlymart/go-textseg/v13 v13.0.0 h1:Y+KvPE1NYz0xl601PVImeQfFyEy6iT90AvPUL1NNfNw= github.com/apparentlymart/go-textseg/v13 v13.0.0/go.mod h1:ZK2fH7c4NqDTLtiYLvIkEghdlcqw7yxLeM89kiTRPUo= github.com/aws/aws-sdk-go v1.15.78/go.mod h1:E3/ieXAlvM0XWO57iftYVDLLvQ824smPP3ATZkfNZeM= github.com/aws/aws-sdk-go v1.40.56 h1:FM2yjR0UUYFzDTMx+mH9Vyw1k1EUUxsAFzk+BjkzANA= github.com/aws/aws-sdk-go v1.40.56/go.mod h1:585smgzpB/KqRA+K3y/NL/oYRqQvpNJYvLm+LY1U59Q= github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d h1:xDfNPAt8lFiC1UJrqV3uuy861HCTo708pDMbjHHdCas= github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d/go.mod h1:6QX/PXZ00z/TKoufEY6K/a0k6AhaJrQKdFe6OfVXsa4= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= github.com/cheggaaa/pb v1.0.27/go.mod h1:pQciLPpbU0oxA0h+VJYYLxO+XeDQb5pZijXscXHm81s= github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po= github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= github.com/envoyproxy/go-control-plane v0.9.9-0.20210217033140-668b12f5399d/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4= github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= github.com/go-test/deep v1.0.3/go.mod h1:wGDj63lr65AM2AQyKZd/NYHGb0R+1RLqB8NKt3aSFNA= github.com/go-test/deep v1.0.7 h1:/VSMRlnY/JSyqxQUzQLKVMAskpY/NZKFA5j2P+0pP2M= github.com/go-test/deep v1.0.7/go.mod h1:QV8Hv/iy04NyLBxAdO9njL0iVPN1S4d/A3NVv1V36o8= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e h1:1r7pUrabqp18hOBcwBwiTsbnFeTZHV9eER/QT5JVZxY= github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y= github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4= github.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8= github.com/golang/protobuf v1.1.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk= github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= github.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM= github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw= github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= github.com/golang/snappy v0.0.3 h1:fHPg5GQYlCeLIPB9BZqMVR5nR9A+IM5zcgeTdjMYmLA= github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.7 h1:81/ik6ipDQS2aGcBfIN5dHDB36BwrStyeAQquSYCV4o= github.com/google/martian v2.1.0+incompatible h1:/CP5g8u/VJHijgedC/Legn3BAbAaWPgecwXBIDzw5no= github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= github.com/google/martian/v3 v3.2.1 h1:d8MncMlErDFTwQGBK1xhv026j9kqhvw1Qv9IbWT1VLQ= github.com/google/martian/v3 v3.2.1/go.mod h1:oBOf6HBosgwRXnUGWUB05QECsc6uvmMiJ3+6W4l/CUk= github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/pprof v0.0.0-20210601050228-01bbb1931b22/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE= github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= github.com/googleapis/gax-go/v2 v2.0.5 h1:sjZBwGj9Jlw33ImPtvFviGYvseOtDM7hkSKB7+Tv3SM= github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= github.com/gruntwork-io/terratest v0.40.23 h1:UKSJhrXfbyiaGOkQmqjTtbQsXi+9uSu3H8nrT9X1PGg= github.com/gruntwork-io/terratest v0.40.23/go.mod h1:JGeIGgLbxbG9/Oqm06z6YXVr76CfomdmLkV564qov+8= github.com/hashicorp/errwrap v1.0.0 h1:hLrqtEDnRye3+sgx6z4qVLNuviH3MR5aQ0ykNJa/UYA= github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ= github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48= github.com/hashicorp/go-getter v1.6.1 h1:NASsgP4q6tL94WH6nJxKWj8As2H/2kop/bB1d8JMyRY= github.com/hashicorp/go-getter v1.6.1/go.mod h1:IZCrswsZPeWv9IkVnLElzRU/gz/QPi6pZHn4tv6vbwA= github.com/hashicorp/go-multierror v1.1.0 h1:B9UzwGQJehnUY1yNrnwREHc3fGbC2xefo8g4TbElacI= github.com/hashicorp/go-multierror v1.1.0/go.mod h1:spPvp8C1qA32ftKqdAHm4hHTbPw+vmowP0z+KUhOZdA= github.com/hashicorp/go-safetemp v1.0.0 h1:2HR189eFNrjHQyENnQMMpCiBAsRxzbTMIgBhEyExpmo= github.com/hashicorp/go-safetemp v1.0.0/go.mod h1:oaerMy3BhqiTbVye6QuFhFtIceqFoDHxNAB65b+Rj1I= github.com/hashicorp/go-version v1.1.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA= github.com/hashicorp/go-version v1.3.0 h1:McDWVJIU/y+u1BRV06dPaLfLCaT7fUTJLp5r04x7iNw= github.com/hashicorp/go-version v1.3.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA= github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/hcl/v2 v2.9.1 h1:eOy4gREY0/ZQHNItlfuEZqtcQbXIxzojlP301hDpnac= github.com/hashicorp/hcl/v2 v2.9.1/go.mod h1:FwWsfWEjyV/CMj8s/gqAuiviY72rJ1/oayI9WftqcKg= github.com/hashicorp/terraform-json v0.13.0 h1:Li9L+lKD1FO5RVFRM1mMMIBDoUHslOniyEi5CM+FWGY= github.com/hashicorp/terraform-json v0.13.0/go.mod h1:y5OdLBCT+rxbwnpxZs9kGL7R9ExU76+cpdY8zHwoazk= github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= github.com/jinzhu/copier v0.0.0-20190924061706-b57f9002281a h1:zPPuIq2jAWWPTrGt70eK/BSch+gFAGrNzecsoENgu2o= github.com/jinzhu/copier v0.0.0-20190924061706-b57f9002281a/go.mod h1:yL958EeXv8Ylng6IfnvG4oflryUi3vgA3xPs9hmII1s= github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg= github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo= github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8= github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U= github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= github.com/jstemmer/go-junit-report v0.9.1 h1:6QPYqodiu3GuPL+7mfx+NwDdp2eTkp9IfEUpgAwUN0o= github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= github.com/klauspost/compress v1.11.2/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs= github.com/klauspost/compress v1.13.0 h1:2T7tUoQrQT+fQWdaY5rjWztFGAFwbGD04iPJg90ZiOs= github.com/klauspost/compress v1.13.0/go.mod h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.2.1 h1:Fmg33tUaq4/8ym9TJN1x7sLJnHVwhP33CNkpYV/7rwI= github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/kylelemons/godebug v0.0.0-20170820004349-d65d576e9348/go.mod h1:B69LEHPfb2qLo0BaaOLcbitczOKLWTsrBG9LczfCD4k= github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU= github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4= github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU= github.com/mattn/go-zglob v0.0.2-0.20190814121620-e3c945676326 h1:ofNAzWCcyTALn2Zv40+8XitdzCgXY6e9qvXwN9W0YXg= github.com/mattn/go-zglob v0.0.2-0.20190814121620-e3c945676326/go.mod h1:9fxibJccNxU2cnpIKLRRFA7zX7qhkJIQWBb449FYHOo= github.com/mitchellh/copystructure v1.2.0/go.mod h1:qLl+cE2AmVv+CoeAwDPye/v+N2HKCj9FbZEVFJRxO9s= github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y= github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= github.com/mitchellh/go-testing-interface v1.0.0 h1:fzU/JVNcaqHQEcVFAKeR41fkiLdIPrefOvVG1VZ96U0= github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI= github.com/mitchellh/go-wordwrap v0.0.0-20150314170334-ad45545899c7/go.mod h1:ZXFpozHsX6DPmq2I0TCekCxypsnAUbP2oI0UX1GXzOo= github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQflz0v0= github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0= github.com/mitchellh/reflectwalk v1.0.2/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= github.com/sebdah/goldie v1.0.0/go.mod h1:jXP4hmWywNEwZzhMuv2ccnqTSFpuq8iyQhtQdkkZBH4= github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo= github.com/spf13/pflag v1.0.2/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/tmccombs/hcl2json v0.3.3 h1:+DLNYqpWE0CsOQiEZu+OZm5ZBImake3wtITYxQ8uLFQ= github.com/tmccombs/hcl2json v0.3.3/go.mod h1:Y2chtz2x9bAeRTvSibVRVgbLJhLJXKlUeIvjeVdnm4w= github.com/ulikunitz/xz v0.5.8 h1:ERv8V6GKqVi23rgu5cj9pVfVzJbOqAY2Ntl88O6c2nQ= github.com/ulikunitz/xz v0.5.8/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14= github.com/vmihailenco/msgpack v3.3.3+incompatible/go.mod h1:fy3FlTQTDXWkZ7Bh6AcGMlsjHatGryHQYUTf1ShIgkk= github.com/vmihailenco/msgpack/v4 v4.3.12/go.mod h1:gborTTJjAo/GWTqqRjrLCn9pgNN+NXzzngzBKDPIqw4= github.com/vmihailenco/tagparser v0.1.1/go.mod h1:OeAg3pn3UbLjkWt+rN9oFYB6u/cQgqMEUPoW2WPyhdI= github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= github.com/zclconf/go-cty v1.2.0/go.mod h1:hOPWgoHbaTUnI5k4D2ld+GRpFJSCe6bCM7m1q/N4PQ8= github.com/zclconf/go-cty v1.8.0/go.mod h1:vVKLxnk3puL4qRAv72AO+W99LUD4da90g3uUAzyuvAk= github.com/zclconf/go-cty v1.8.1/go.mod h1:vVKLxnk3puL4qRAv72AO+W99LUD4da90g3uUAzyuvAk= github.com/zclconf/go-cty v1.9.1 h1:viqrgQwFl5UpSxc046qblj78wZXVDFnSOufaOTER+cc= github.com/zclconf/go-cty v1.9.1/go.mod h1:vVKLxnk3puL4qRAv72AO+W99LUD4da90g3uUAzyuvAk= github.com/zclconf/go-cty-debug v0.0.0-20191215020915-b22d67c1ba0b/go.mod h1:ZRKQfBXbGkpdV6QMzT3rU1kSTAnfu1dO8dPKjYprgj8= go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk= go.opencensus.io v0.23.0 h1:gqCw0LfLxScz8irSi8exQc7fyQ0fKQU/qnC/X8+V/1M= go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190426145343-a29dc8fdc734/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a h1:kr2P4QFmQr29mSLA43kwrOcgcReGTfbE9N577tCTuBc= golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a/go.mod h1:P+XmwS30IXTQdn5tA2iutPOUgjI07+tq3H3K9MVA1s8= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek= golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY= golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM= golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU= golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs= golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= golang.org/x/lint v0.0.0-20210508222113-6edffad5e616 h1:VLliZ0d+/avPrXXH+OakdXhpJuEoBZuwh1m2j7U6Iug= golang.org/x/lint v0.0.0-20210508222113-6edffad5e616/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE= golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o= golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY= golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.4.2 h1:Gz96sIWK3OalVv/I/qNygP42zyoKp3xptRVCWRFEBvo= golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180811021610-c39426892332/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc= golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= golang.org/x/net v0.0.0-20210503060351-7fd8e65b6420/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20210614182718-04defd469f4e h1:XpT3nA5TvE525Ne3hInMh6+GETgn27Zfm9dxsThnX2Q= golang.org/x/net v0.0.0-20210614182718-04defd469f4e/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c h1:pkQiBZBvdos9qq4wBAHqlzuZHEXo07pqV06ef90u1WI= golang.org/x/oauth2 v0.0.0-20210514164344-f6687ab2804c/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190502175342-a43fa875dd82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210514084401-e8d321eab015/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220517195934-5e4e11fc645e h1:w36l2Uw3dRan1K3TyXriXvY+6T56GNmlKGcqiQUJDfM= golang.org/x/sys v0.0.0-20220517195934-5e4e11fc645e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 h1:v+OssWQX+hTHEmOBgwxdZxK4zHq3yOs8F9J7mk0PY8E= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.6 h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M= golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8= golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE= golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= golang.org/x/tools v0.1.1/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/tools v0.1.2 h1:kRBLX7v7Af8W7Gdbbc908OJcdgtK8bOz9Uaj8/F1ACA= golang.org/x/tools v0.1.2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE= google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M= google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM= google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc= google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg= google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE= google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8= google.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU= google.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94= google.golang.org/api v0.47.0 h1:sQLWZQvP6jPGIP4JGPkJu4zHswrv81iobiyszr3b/0I= google.golang.org/api v0.47.0/go.mod h1:Wbvgpq1HddcWVtzsVLyfLp8lDg6AA241LmgIL59tHXo= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0= google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c= google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8= google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA= google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U= google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA= google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A= google.golang.org/genproto v0.0.0-20210513213006-bf773b8c8384/go.mod h1:P3QM42oQyzQSnHPnZ/vqoCdDmzH28fzWByN9asMeM8A= google.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c h1:wtujag7C+4D6KMoulW9YauvK2lgdvCMS260jsqqBXr0= google.golang.org/genproto v0.0.0-20210602131652-f16073e35f0c/go.mod h1:UODoCrxHCcBojKKwX1terBiRUaqAsFqJiF615XL43r0= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38= google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY= google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60= google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk= google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= google.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc= google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8= google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= google.golang.org/grpc v1.36.1/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU= google.golang.org/grpc v1.37.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM= google.golang.org/grpc v1.37.1/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM= google.golang.org/grpc v1.38.0 h1:/9BgsAsa5nWe26HqOlvlgJnqBuktYOLCgjCPqsa56W0= google.golang.org/grpc v1.38.0/go.mod h1:NREThFqKR1f3iQ6oBuvc5LadQuXVGo9rkm5ZGrQdJfM= google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0/go.mod h1:6Kw0yEErY5E/yWrBtf03jp27GLLJujG4z/JK95pnjjw= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4= google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.26.0 h1:bxAC2xTBsZGibn2RTntX0oH50xLsqy1OxA9tTL3p/lk= google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/cheggaaa/pb.v1 v1.0.27/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw= gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b h1:h8qDotaEPuJATrMmW04NCwg7v22aHH28wwpauUhK9Oo= gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= ================================================ FILE: modules/README.md ================================================ # swiss-army-kube modules ### Every new module firstly will be added to our [sak-incubator](https://github.com/provectus/sak-incubator). Unreleased sak-modules are here too. When the module will be ready, we will release it as separate sak-module. ### If you are interested in our project and want to see your Kubernetes tool as SAK-module - please create an [request](https://github.com/provectus/swiss-army-kube/issues) in GitHub. ## Infrastructure * [sak-kubernetes](https://github.com/provectus/sak-kubernetes) - Bootstrap EKS cluster, based on [terraform-aws-eks](https://github.com/terraform-aws-modules/terraform-aws-eks) module. * [sak-vpc](https://github.com/provectus/sak-vpc) - Prepare VPC and networking for EKS cluster and nodes, based on [terraform-aws-vpc](https://github.com/terraform-aws-modules/terraform-aws-vpc) module. * [sak-argocd](https://github.com/provectus/sak-argocd) - Argocd deployment, which will be used as main controller of sak-modules configuration. ## Controllers * [sak-alb-controller](https://github.com/provectus/sak-alb-controller) - Create Elastic Load Balancers in EKS. * [sak-external-dns](https://github.com/provectus/sak-external-dns) - ExternalDNS synchronizes exposed Kubernetes Services and Ingresses with DNS providers(Route53). * [sak-scaling](https://github.com/provectus/sak-scaling) - Consist of Node Autoscaler and Horizontal Pod Autoscaler. * [sak-cert-manager](https://github.com/provectus/sak-cert-manager) - X.509 certificate management for Kubernetes. * [sak-nginx](https://github.com/provectus/sak-nginx) - Ingress controller for Kubernetes using NGINX as a reverse proxy and load balancer. * [sak-external-secrets](https://github.com/provectus/sak-external-secrets) - A Kubernetes operator that integrates external secret management systems like AWS Secrets Manager. ## Monitoring/Observability * [sak-loki](https://github.com/provectus/sak-loki) - [Grafana Loki](https://grafana.com/oss/loki/), log aggregation and processing system. * [sak-prometheus](https://github.com/provectus/sak-prometheus) - Prometheus, systems and service monitoring system. * [sak-efk](https://github.com/provectus/sak-efk) - ElasticSearch + Filebeat + Kibana stack. * [sak-victoria-metrics](https://github.com/provectus/sak-victoria-metrics) - Deployment of [Victoria Metrics](https://victoriametrics.com/). ## Authentication * [sak-oauth](https://github.com/provectus/sak-oauth) - Deployment of [Oauth proxy](https://github.com/oauth2-proxy/oauth2-proxy). * [sak-cognito](https://github.com/provectus/sak-cognito) - Integration of [AWS Cognito](https://aws.amazon.com/ru/cognito/). ## Specific modules * [sak-kubeflow](https://github.com/provectus/sak-kubeflow) - Make EKS cluster ML-Ready, using [Kubeflow](https://www.kubeflow.org/). ================================================ FILE: prerequisites_install.sh ================================================ #!/bin/bash SCRIPTPATH="$( cd "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )" brew install helm kubernetes-cli awscli aws-iam-authenticator terraform jq echo echo "Installed helm version:" helm version echo echo "Installed kubectl version:" kubectl version --client echo echo "Installed awscli version:" aws --version echo echo "Installed aws-iam-authenticator version:" aws-iam-authenticator version echo echo "Installed terraform version:" terraform version | head -n 1 echo echo "Installed jq version:" jq --version echo