Showing preview only (3,547K chars total). Download the full file or copy to clipboard to get everything.
Repository: kubernetes-sigs/kubespray
Branch: master
Commit: 341da0e8cb8c
Files: 1193
Total size: 3.1 MB
Directory structure:
gitextract_cnwkyuho/
├── .ansible-lint
├── .ansible-lint-ignore
├── .editorconfig
├── .gitattributes
├── .github/
│ ├── ISSUE_TEMPLATE/
│ │ ├── bug-report.yaml
│ │ ├── config.yml
│ │ ├── enhancement.yaml
│ │ └── failing-test.yaml
│ ├── PULL_REQUEST_TEMPLATE.md
│ ├── dependabot.yml
│ └── workflows/
│ ├── auto-label-os.yml
│ ├── upgrade-patch-versions-schedule.yml
│ └── upgrade-patch-versions.yml
├── .gitignore
├── .gitlab-ci/
│ ├── build.yml
│ ├── kubevirt.yml
│ ├── lint.yml
│ ├── molecule.yml
│ ├── terraform.yml
│ └── vagrant.yml
├── .gitlab-ci.yml
├── .gitmodules
├── .md_style.rb
├── .mdlrc
├── .nojekyll
├── .pre-commit-config.yaml
├── .yamllint
├── CHANGELOG.md
├── CNAME
├── CONTRIBUTING.md
├── Dockerfile
├── LICENSE
├── OWNERS
├── OWNERS_ALIASES
├── README.md
├── RELEASE.md
├── SECURITY_CONTACTS
├── Vagrantfile
├── _config.yml
├── ansible.cfg
├── cluster.yml
├── code-of-conduct.md
├── contrib/
│ ├── aws_iam/
│ │ ├── kubernetes-master-policy.json
│ │ ├── kubernetes-master-role.json
│ │ ├── kubernetes-minion-policy.json
│ │ └── kubernetes-minion-role.json
│ ├── aws_inventory/
│ │ ├── kubespray-aws-inventory.py
│ │ └── requirements.txt
│ ├── azurerm/
│ │ ├── .gitignore
│ │ ├── README.md
│ │ ├── apply-rg.sh
│ │ ├── clear-rg.sh
│ │ ├── generate-inventory.sh
│ │ ├── generate-inventory.yml
│ │ ├── generate-inventory_2.yml
│ │ ├── generate-templates.yml
│ │ ├── group_vars/
│ │ │ └── all
│ │ └── roles/
│ │ ├── generate-inventory/
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ └── inventory.j2
│ │ ├── generate-inventory_2/
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ ├── inventory.j2
│ │ │ └── loadbalancer_vars.j2
│ │ └── generate-templates/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ ├── tasks/
│ │ │ └── main.yml
│ │ └── templates/
│ │ ├── availability-sets.json
│ │ ├── bastion.json
│ │ ├── clear-rg.json
│ │ ├── masters.json
│ │ ├── minions.json
│ │ ├── network.json
│ │ └── storage.json
│ ├── collection.sh
│ ├── offline/
│ │ ├── README.md
│ │ ├── docker-daemon.json
│ │ ├── generate_list.sh
│ │ ├── generate_list.yml
│ │ ├── manage-offline-container-images.sh
│ │ ├── manage-offline-files.sh
│ │ ├── nginx.conf
│ │ ├── registries.conf
│ │ └── upload2artifactory.py
│ ├── os-services/
│ │ ├── os-services.yml
│ │ └── roles/
│ │ └── prepare/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ └── tasks/
│ │ └── main.yml
│ └── terraform/
│ ├── aws/
│ │ ├── .gitignore
│ │ ├── README.md
│ │ ├── create-infrastructure.tf
│ │ ├── credentials.tfvars.example
│ │ ├── modules/
│ │ │ ├── iam/
│ │ │ │ ├── main.tf
│ │ │ │ ├── outputs.tf
│ │ │ │ └── variables.tf
│ │ │ ├── nlb/
│ │ │ │ ├── main.tf
│ │ │ │ ├── outputs.tf
│ │ │ │ └── variables.tf
│ │ │ └── vpc/
│ │ │ ├── main.tf
│ │ │ ├── outputs.tf
│ │ │ └── variables.tf
│ │ ├── output.tf
│ │ ├── sample-inventory/
│ │ │ └── cluster.tfvars
│ │ ├── templates/
│ │ │ └── inventory.tpl
│ │ ├── terraform.tfvars
│ │ ├── terraform.tfvars.example
│ │ └── variables.tf
│ ├── exoscale/
│ │ ├── README.md
│ │ ├── default.tfvars
│ │ ├── main.tf
│ │ ├── modules/
│ │ │ └── kubernetes-cluster/
│ │ │ ├── main.tf
│ │ │ ├── output.tf
│ │ │ ├── templates/
│ │ │ │ └── cloud-init.tmpl
│ │ │ ├── variables.tf
│ │ │ └── versions.tf
│ │ ├── output.tf
│ │ ├── sample-inventory/
│ │ │ └── cluster.tfvars
│ │ ├── templates/
│ │ │ └── inventory.tpl
│ │ ├── variables.tf
│ │ └── versions.tf
│ ├── gcp/
│ │ ├── README.md
│ │ ├── generate-inventory.sh
│ │ ├── main.tf
│ │ ├── modules/
│ │ │ └── kubernetes-cluster/
│ │ │ ├── main.tf
│ │ │ ├── output.tf
│ │ │ └── variables.tf
│ │ ├── output.tf
│ │ ├── tfvars.json
│ │ └── variables.tf
│ ├── hetzner/
│ │ ├── README.md
│ │ ├── default.tfvars
│ │ ├── main.tf
│ │ ├── modules/
│ │ │ ├── kubernetes-cluster/
│ │ │ │ ├── main.tf
│ │ │ │ ├── output.tf
│ │ │ │ ├── templates/
│ │ │ │ │ └── cloud-init.tmpl
│ │ │ │ ├── variables.tf
│ │ │ │ └── versions.tf
│ │ │ └── kubernetes-cluster-flatcar/
│ │ │ ├── main.tf
│ │ │ ├── outputs.tf
│ │ │ ├── templates/
│ │ │ │ └── machine.yaml.tmpl
│ │ │ ├── variables.tf
│ │ │ └── versions.tf
│ │ ├── output.tf
│ │ ├── sample-inventory/
│ │ │ └── cluster.tfvars
│ │ ├── templates/
│ │ │ └── inventory.tpl
│ │ ├── variables.tf
│ │ └── versions.tf
│ ├── openstack/
│ │ ├── .gitignore
│ │ ├── README.md
│ │ ├── kubespray.tf
│ │ ├── modules/
│ │ │ ├── compute/
│ │ │ │ ├── ansible_bastion_template.txt
│ │ │ │ ├── main.tf
│ │ │ │ ├── outputs.tf
│ │ │ │ ├── templates/
│ │ │ │ │ └── cloudinit.yaml.tmpl
│ │ │ │ ├── variables.tf
│ │ │ │ └── versions.tf
│ │ │ ├── ips/
│ │ │ │ ├── main.tf
│ │ │ │ ├── outputs.tf
│ │ │ │ ├── variables.tf
│ │ │ │ └── versions.tf
│ │ │ ├── loadbalancer/
│ │ │ │ ├── main.tf
│ │ │ │ ├── variables.tf
│ │ │ │ └── versions.tf
│ │ │ └── network/
│ │ │ ├── main.tf
│ │ │ ├── outputs.tf
│ │ │ ├── variables.tf
│ │ │ └── versions.tf
│ │ ├── variables.tf
│ │ └── versions.tf
│ ├── terraform.py
│ ├── upcloud/
│ │ ├── README.md
│ │ ├── cluster-settings.tfvars
│ │ ├── main.tf
│ │ ├── modules/
│ │ │ └── kubernetes-cluster/
│ │ │ ├── main.tf
│ │ │ ├── output.tf
│ │ │ ├── variables.tf
│ │ │ └── versions.tf
│ │ ├── output.tf
│ │ ├── sample-inventory/
│ │ │ └── cluster.tfvars
│ │ ├── templates/
│ │ │ └── inventory.tpl
│ │ ├── variables.tf
│ │ └── versions.tf
│ └── vsphere/
│ ├── README.md
│ ├── default.tfvars
│ ├── main.tf
│ ├── modules/
│ │ └── kubernetes-cluster/
│ │ ├── main.tf
│ │ ├── output.tf
│ │ ├── templates/
│ │ │ ├── cloud-init.tpl
│ │ │ ├── metadata.tpl
│ │ │ └── vapp-cloud-init.tpl
│ │ ├── variables.tf
│ │ └── versions.tf
│ ├── output.tf
│ ├── sample-inventory/
│ │ └── cluster.tfvars
│ ├── templates/
│ │ └── inventory.tpl
│ ├── variables.tf
│ └── versions.tf
├── docs/
│ ├── CNI/
│ │ ├── calico.md
│ │ ├── cilium.md
│ │ ├── cni.md
│ │ ├── flannel.md
│ │ ├── kube-ovn.md
│ │ ├── kube-router.md
│ │ ├── macvlan.md
│ │ └── multus.md
│ ├── CRI/
│ │ ├── containerd.md
│ │ ├── cri-o.md
│ │ ├── docker.md
│ │ ├── gvisor.md
│ │ └── kata-containers.md
│ ├── CSI/
│ │ ├── aws-ebs-csi.md
│ │ ├── azure-csi.md
│ │ ├── cinder-csi.md
│ │ ├── gcp-pd-csi.md
│ │ └── vsphere-csi.md
│ ├── _sidebar.md
│ ├── advanced/
│ │ ├── arch.md
│ │ ├── cert_manager.md
│ │ ├── dns-stack.md
│ │ ├── downloads.md
│ │ ├── gcp-lb.md
│ │ ├── kubernetes-reliability.md
│ │ ├── netcheck.md
│ │ ├── ntp.md
│ │ ├── proxy.md
│ │ └── registry.md
│ ├── ansible/
│ │ ├── ansible.md
│ │ ├── ansible_collection.md
│ │ ├── inventory.md
│ │ └── vars.md
│ ├── calico_peer_example/
│ │ ├── new-york.yml
│ │ └── paris.yml
│ ├── cloud_controllers/
│ │ ├── openstack.md
│ │ └── vsphere.md
│ ├── cloud_providers/
│ │ ├── aws.md
│ │ ├── azure.md
│ │ └── cloud.md
│ ├── developers/
│ │ ├── ci-setup.md
│ │ ├── ci.md
│ │ ├── test_cases.md
│ │ └── vagrant.md
│ ├── external_storage_provisioners/
│ │ ├── local_volume_provisioner.md
│ │ └── scheduler_plugins.md
│ ├── getting_started/
│ │ ├── comparisons.md
│ │ ├── getting-started.md
│ │ └── setting-up-your-first-cluster.md
│ ├── ingress/
│ │ ├── alb_ingress_controller.md
│ │ ├── kube-vip.md
│ │ └── metallb.md
│ ├── operating_systems/
│ │ ├── amazonlinux.md
│ │ ├── bootstrap-os.md
│ │ ├── fcos.md
│ │ ├── flatcar.md
│ │ ├── kylinlinux.md
│ │ ├── openeuler.md
│ │ ├── opensuse.md
│ │ ├── rhel.md
│ │ └── uoslinux.md
│ ├── operations/
│ │ ├── cgroups.md
│ │ ├── encrypting-secret-data-at-rest.md
│ │ ├── etcd.md
│ │ ├── ha-mode.md
│ │ ├── hardening.md
│ │ ├── integration.md
│ │ ├── kernel-requirements.md
│ │ ├── large-deployments.md
│ │ ├── mirror.md
│ │ ├── nodes.md
│ │ ├── offline-environment.md
│ │ ├── port-requirements.md
│ │ ├── recover-control-plane.md
│ │ └── upgrades.md
│ ├── roadmap/
│ │ └── roadmap.md
│ └── upgrades/
│ └── migrate_docker2containerd.md
├── extra_playbooks/
│ ├── files/
│ │ └── get_cinder_pvs.sh
│ ├── migrate_openstack_provider.yml
│ ├── upgrade-only-k8s.yml
│ └── wait-for-cloud-init.yml
├── galaxy.yml
├── index.html
├── inventory/
│ ├── local/
│ │ └── hosts.ini
│ └── sample/
│ ├── group_vars/
│ │ ├── all/
│ │ │ ├── all.yml
│ │ │ ├── aws.yml
│ │ │ ├── azure.yml
│ │ │ ├── containerd.yml
│ │ │ ├── coreos.yml
│ │ │ ├── cri-o.yml
│ │ │ ├── docker.yml
│ │ │ ├── etcd.yml
│ │ │ ├── gcp.yml
│ │ │ ├── hcloud.yml
│ │ │ ├── huaweicloud.yml
│ │ │ ├── oci.yml
│ │ │ ├── offline.yml
│ │ │ ├── openstack.yml
│ │ │ ├── upcloud.yml
│ │ │ └── vsphere.yml
│ │ └── k8s_cluster/
│ │ ├── addons.yml
│ │ ├── k8s-cluster.yml
│ │ ├── k8s-net-calico.yml
│ │ ├── k8s-net-cilium.yml
│ │ ├── k8s-net-custom-cni.yml
│ │ ├── k8s-net-flannel.yml
│ │ ├── k8s-net-kube-ovn.yml
│ │ ├── k8s-net-kube-router.yml
│ │ ├── k8s-net-macvlan.yml
│ │ └── kube_control_plane.yml
│ └── inventory.ini
├── logo/
│ ├── LICENSE
│ └── usage_guidelines.md
├── meta/
│ └── runtime.yml
├── pipeline.Dockerfile
├── playbooks/
│ ├── ansible_version.yml
│ ├── boilerplate.yml
│ ├── cluster.yml
│ ├── facts.yml
│ ├── install_etcd.yml
│ ├── internal_facts.yml
│ ├── recover_control_plane.yml
│ ├── remove_node.yml
│ ├── reset.yml
│ ├── scale.yml
│ └── upgrade_cluster.yml
├── plugins/
│ └── modules/
│ └── kube.py
├── recover-control-plane.yml
├── remove-node.yml
├── remove_node.yml
├── requirements.txt
├── reset.yml
├── roles/
│ ├── adduser/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ ├── molecule/
│ │ │ └── default/
│ │ │ ├── converge.yml
│ │ │ └── molecule.yml
│ │ ├── tasks/
│ │ │ └── main.yml
│ │ └── vars/
│ │ ├── coreos.yml
│ │ ├── debian.yml
│ │ └── redhat.yml
│ ├── bastion-ssh-config/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ ├── molecule/
│ │ │ └── default/
│ │ │ ├── converge.yml
│ │ │ └── molecule.yml
│ │ ├── tasks/
│ │ │ └── main.yml
│ │ └── templates/
│ │ └── ssh-bastion.conf.j2
│ ├── bootstrap-os/
│ │ └── tasks/
│ │ └── main.yml
│ ├── bootstrap_os/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ ├── files/
│ │ │ └── bootstrap.sh
│ │ ├── handlers/
│ │ │ └── main.yml
│ │ ├── meta/
│ │ │ └── main.yml
│ │ ├── molecule/
│ │ │ └── default/
│ │ │ ├── converge.yml
│ │ │ ├── molecule.yml
│ │ │ └── tests/
│ │ │ └── test_default.py
│ │ ├── tasks/
│ │ │ ├── almalinux.yml
│ │ │ ├── amzn.yml
│ │ │ ├── centos.yml
│ │ │ ├── debian.yml
│ │ │ ├── fedora-coreos.yml
│ │ │ ├── fedora.yml
│ │ │ ├── flatcar.yml
│ │ │ ├── main.yml
│ │ │ ├── openEuler.yml
│ │ │ ├── opensuse-leap.yml
│ │ │ ├── opensuse-tumbleweed.yml
│ │ │ ├── opensuse.yml
│ │ │ ├── rhel.yml
│ │ │ ├── rocky.yml
│ │ │ └── ubuntu.yml
│ │ └── vars/
│ │ ├── fedora-coreos.yml
│ │ └── flatcar.yml
│ ├── container-engine/
│ │ ├── containerd/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── handlers/
│ │ │ │ ├── main.yml
│ │ │ │ └── reset.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── molecule/
│ │ │ │ └── default/
│ │ │ │ ├── converge.yml
│ │ │ │ ├── molecule.yml
│ │ │ │ └── verify.yml
│ │ │ ├── tasks/
│ │ │ │ ├── main.yml
│ │ │ │ └── reset.yml
│ │ │ └── templates/
│ │ │ ├── config-v1.toml.j2
│ │ │ ├── config.toml.j2
│ │ │ ├── containerd.service.j2
│ │ │ ├── hosts.toml.j2
│ │ │ └── http-proxy.conf.j2
│ │ ├── containerd-common/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── vars/
│ │ │ ├── amazon.yml
│ │ │ └── suse.yml
│ │ ├── cri-dockerd/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── handlers/
│ │ │ │ └── main.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── molecule/
│ │ │ │ └── default/
│ │ │ │ ├── converge.yml
│ │ │ │ ├── files/
│ │ │ │ │ ├── 10-mynet.conf
│ │ │ │ │ ├── container.json
│ │ │ │ │ └── sandbox.json
│ │ │ │ ├── molecule.yml
│ │ │ │ └── verify.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ ├── cri-dockerd.service.j2
│ │ │ └── cri-dockerd.socket.j2
│ │ ├── cri-o/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── handlers/
│ │ │ │ └── main.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── molecule/
│ │ │ │ └── default/
│ │ │ │ ├── converge.yml
│ │ │ │ ├── molecule.yml
│ │ │ │ └── verify.yml
│ │ │ ├── tasks/
│ │ │ │ ├── load_vars.yml
│ │ │ │ ├── main.yaml
│ │ │ │ ├── reset.yml
│ │ │ │ └── setup-amazon.yaml
│ │ │ ├── templates/
│ │ │ │ ├── config.json.j2
│ │ │ │ ├── crio.conf.j2
│ │ │ │ ├── http-proxy.conf.j2
│ │ │ │ ├── mounts.conf.j2
│ │ │ │ ├── registry.conf.j2
│ │ │ │ └── unqualified.conf.j2
│ │ │ └── vars/
│ │ │ ├── v1.29.yml
│ │ │ └── v1.31.yml
│ │ ├── crictl/
│ │ │ ├── handlers/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ └── crictl.yaml.j2
│ │ ├── crun/
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── docker/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── files/
│ │ │ │ └── cleanup-docker-orphans.sh
│ │ │ ├── handlers/
│ │ │ │ └── main.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ ├── docker_plugin.yml
│ │ │ │ ├── main.yml
│ │ │ │ ├── pre-upgrade.yml
│ │ │ │ ├── reset.yml
│ │ │ │ ├── set_facts_dns.yml
│ │ │ │ └── systemd.yml
│ │ │ ├── templates/
│ │ │ │ ├── docker-dns.conf.j2
│ │ │ │ ├── docker-options.conf.j2
│ │ │ │ ├── docker-orphan-cleanup.conf.j2
│ │ │ │ ├── docker.service.j2
│ │ │ │ ├── fedora_docker.repo.j2
│ │ │ │ ├── http-proxy.conf.j2
│ │ │ │ └── rh_docker.repo.j2
│ │ │ └── vars/
│ │ │ ├── amazon.yml
│ │ │ ├── clearlinux.yml
│ │ │ ├── debian.yml
│ │ │ ├── fedora.yml
│ │ │ ├── kylin.yml
│ │ │ ├── redhat.yml
│ │ │ ├── suse.yml
│ │ │ ├── ubuntu.yml
│ │ │ └── uniontech.yml
│ │ ├── gvisor/
│ │ │ ├── molecule/
│ │ │ │ └── default/
│ │ │ │ ├── converge.yml
│ │ │ │ ├── files/
│ │ │ │ │ ├── 10-mynet.conf
│ │ │ │ │ ├── container.json
│ │ │ │ │ └── sandbox.json
│ │ │ │ ├── molecule.yml
│ │ │ │ └── verify.yml
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── kata-containers/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── molecule/
│ │ │ │ └── default/
│ │ │ │ ├── converge.yml
│ │ │ │ ├── files/
│ │ │ │ │ ├── 10-mynet.conf
│ │ │ │ │ ├── container.json
│ │ │ │ │ └── sandbox.json
│ │ │ │ ├── molecule.yml
│ │ │ │ └── verify.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ ├── configuration-qemu.toml.j2
│ │ │ └── containerd-shim-kata-v2.j2
│ │ ├── molecule/
│ │ │ ├── files/
│ │ │ │ └── 10-mynet.conf
│ │ │ ├── prepare.yml
│ │ │ ├── templates/
│ │ │ │ ├── container.json.j2
│ │ │ │ └── sandbox.json.j2
│ │ │ ├── test_cri.yml
│ │ │ └── test_runtime.yml
│ │ ├── nerdctl/
│ │ │ ├── handlers/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ └── nerdctl.toml.j2
│ │ ├── runc/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── skopeo/
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── tasks/
│ │ │ └── main.yml
│ │ ├── validate-container-engine/
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ └── youki/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ ├── molecule/
│ │ │ └── default/
│ │ │ ├── converge.yml
│ │ │ ├── files/
│ │ │ │ ├── 10-mynet.conf
│ │ │ │ ├── container.json
│ │ │ │ └── sandbox.json
│ │ │ ├── molecule.yml
│ │ │ └── verify.yml
│ │ └── tasks/
│ │ └── main.yml
│ ├── download/
│ │ ├── meta/
│ │ │ └── main.yml
│ │ ├── tasks/
│ │ │ ├── check_pull_required.yml
│ │ │ ├── download_container.yml
│ │ │ ├── download_file.yml
│ │ │ ├── extract_file.yml
│ │ │ ├── main.yml
│ │ │ ├── prep_download.yml
│ │ │ ├── prep_kubeadm_images.yml
│ │ │ └── set_container_facts.yml
│ │ └── templates/
│ │ └── kubeadm-images.yaml.j2
│ ├── dynamic_groups/
│ │ └── tasks/
│ │ └── main.yml
│ ├── etcd/
│ │ ├── handlers/
│ │ │ ├── backup.yml
│ │ │ ├── backup_cleanup.yml
│ │ │ └── main.yml
│ │ ├── meta/
│ │ │ └── main.yml
│ │ ├── tasks/
│ │ │ ├── check_certs.yml
│ │ │ ├── clean_v2_store.yml
│ │ │ ├── configure.yml
│ │ │ ├── gen_certs_script.yml
│ │ │ ├── gen_nodes_certs_script.yml
│ │ │ ├── install_docker.yml
│ │ │ ├── install_host.yml
│ │ │ ├── join_etcd-events_member.yml
│ │ │ ├── join_etcd_member.yml
│ │ │ ├── main.yml
│ │ │ ├── refresh_config.yml
│ │ │ └── upd_ca_trust.yml
│ │ └── templates/
│ │ ├── etcd-docker.service.j2
│ │ ├── etcd-events-docker.service.j2
│ │ ├── etcd-events-host.service.j2
│ │ ├── etcd-events.env.j2
│ │ ├── etcd-events.j2
│ │ ├── etcd-host.service.j2
│ │ ├── etcd.env.j2
│ │ ├── etcd.j2
│ │ ├── make-ssl-etcd.sh.j2
│ │ └── openssl.conf.j2
│ ├── etcd_defaults/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ └── vars/
│ │ └── main.yml
│ ├── etcdctl_etcdutl/
│ │ ├── tasks/
│ │ │ └── main.yml
│ │ └── templates/
│ │ └── etcdctl.sh.j2
│ ├── helm-apps/
│ │ ├── README.md
│ │ ├── meta/
│ │ │ ├── argument_specs.yml
│ │ │ └── main.yml
│ │ ├── tasks/
│ │ │ └── main.yml
│ │ └── vars/
│ │ └── main.yml
│ ├── kubernetes/
│ │ ├── client/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── control-plane/
│ │ │ ├── defaults/
│ │ │ │ └── main/
│ │ │ │ ├── etcd.yml
│ │ │ │ ├── kube-proxy.yml
│ │ │ │ ├── kube-scheduler.yml
│ │ │ │ └── main.yml
│ │ │ ├── handlers/
│ │ │ │ └── main.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ ├── check-api.yml
│ │ │ │ ├── encrypt-at-rest.yml
│ │ │ │ ├── kubeadm-backup.yml
│ │ │ │ ├── kubeadm-etcd.yml
│ │ │ │ ├── kubeadm-secondary.yml
│ │ │ │ ├── kubeadm-setup.yml
│ │ │ │ ├── kubeadm-upgrade.yml
│ │ │ │ ├── main.yml
│ │ │ │ └── pre-upgrade.yml
│ │ │ ├── templates/
│ │ │ │ ├── admission-controls.yaml.j2
│ │ │ │ ├── apiserver-audit-policy.yaml.j2
│ │ │ │ ├── apiserver-audit-webhook-config.yaml.j2
│ │ │ │ ├── apiserver-tracing.yaml.j2
│ │ │ │ ├── eventratelimit.yaml.j2
│ │ │ │ ├── k8s-certs-renew.service.j2
│ │ │ │ ├── k8s-certs-renew.sh.j2
│ │ │ │ ├── k8s-certs-renew.timer.j2
│ │ │ │ ├── kubeadm-config.v1beta4.yaml.j2
│ │ │ │ ├── kubeadm-controlplane.yaml.j2
│ │ │ │ ├── kubescheduler-config.yaml.j2
│ │ │ │ ├── podnodeselector.yaml.j2
│ │ │ │ ├── podsecurity.yaml.j2
│ │ │ │ ├── resourcequota.yaml.j2
│ │ │ │ ├── secrets_encryption.yaml.j2
│ │ │ │ ├── webhook-authorization-config.yaml.j2
│ │ │ │ └── webhook-token-auth-config.yaml.j2
│ │ │ └── vars/
│ │ │ └── main.yaml
│ │ ├── kubeadm/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── handlers/
│ │ │ │ └── main.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ ├── kubeadm_etcd_node.yml
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ └── kubeadm-client.conf.j2
│ │ ├── kubeadm_common/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── node/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── handlers/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ ├── facts.yml
│ │ │ │ ├── install.yml
│ │ │ │ ├── kubelet.yml
│ │ │ │ ├── loadbalancer/
│ │ │ │ │ ├── haproxy.yml
│ │ │ │ │ ├── kube-vip.yml
│ │ │ │ │ └── nginx-proxy.yml
│ │ │ │ └── main.yml
│ │ │ ├── templates/
│ │ │ │ ├── http-proxy.conf.j2
│ │ │ │ ├── kubelet-config.v1beta1.yaml.j2
│ │ │ │ ├── kubelet.env.v1beta1.j2
│ │ │ │ ├── kubelet.service.j2
│ │ │ │ ├── loadbalancer/
│ │ │ │ │ ├── haproxy.cfg.j2
│ │ │ │ │ └── nginx.conf.j2
│ │ │ │ └── manifests/
│ │ │ │ ├── haproxy.manifest.j2
│ │ │ │ ├── kube-vip.manifest.j2
│ │ │ │ └── nginx-proxy.manifest.j2
│ │ │ └── vars/
│ │ │ ├── fedora.yml
│ │ │ ├── ubuntu-18.yml
│ │ │ ├── ubuntu-20.yml
│ │ │ ├── ubuntu-22.yml
│ │ │ └── ubuntu-24.yml
│ │ ├── node-label/
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── node-taint/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ └── preinstall/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ ├── files/
│ │ │ └── dhclient_nodnsupdate
│ │ ├── handlers/
│ │ │ └── main.yml
│ │ ├── meta/
│ │ │ └── main.yml
│ │ ├── tasks/
│ │ │ ├── 0010-swapoff.yml
│ │ │ ├── 0020-set_facts.yml
│ │ │ ├── 0040-verify-settings.yml
│ │ │ ├── 0050-create_directories.yml
│ │ │ ├── 0060-resolvconf.yml
│ │ │ ├── 0061-systemd-resolved.yml
│ │ │ ├── 0062-networkmanager-unmanaged-devices.yml
│ │ │ ├── 0063-networkmanager-dns.yml
│ │ │ ├── 0080-system-configurations.yml
│ │ │ ├── 0081-ntp-configurations.yml
│ │ │ ├── 0100-dhclient-hooks.yml
│ │ │ ├── 0110-dhclient-hooks-undo.yml
│ │ │ └── main.yml
│ │ ├── templates/
│ │ │ ├── ansible_git.j2
│ │ │ ├── chrony.conf.j2
│ │ │ ├── dhclient_dnsupdate.sh.j2
│ │ │ ├── dhclient_dnsupdate_rh.sh.j2
│ │ │ ├── ntp.conf.j2
│ │ │ ├── resolvconf.j2
│ │ │ └── resolved.conf.j2
│ │ └── vars/
│ │ └── main.yml
│ ├── kubernetes-apps/
│ │ ├── ansible/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ ├── templates/
│ │ │ │ ├── coredns-clusterrole.yml.j2
│ │ │ │ ├── coredns-clusterrolebinding.yml.j2
│ │ │ │ ├── coredns-config.yml.j2
│ │ │ │ ├── coredns-deployment.yml.j2
│ │ │ │ ├── coredns-poddisruptionbudget.yml.j2
│ │ │ │ ├── coredns-sa.yml.j2
│ │ │ │ ├── coredns-svc.yml.j2
│ │ │ │ ├── dns-autoscaler-clusterrole.yml.j2
│ │ │ │ ├── dns-autoscaler-clusterrolebinding.yml.j2
│ │ │ │ ├── dns-autoscaler-sa.yml.j2
│ │ │ │ ├── dns-autoscaler.yml.j2
│ │ │ │ ├── etcd_metrics-endpoints.yml.j2
│ │ │ │ ├── etcd_metrics-service.yml.j2
│ │ │ │ ├── nodelocaldns-config.yml.j2
│ │ │ │ ├── nodelocaldns-daemonset.yml.j2
│ │ │ │ ├── nodelocaldns-sa.yml.j2
│ │ │ │ └── nodelocaldns-second-daemonset.yml.j2
│ │ │ └── vars/
│ │ │ └── main.yml
│ │ ├── argocd/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ └── argocd-namespace.yml.j2
│ │ ├── cluster_roles/
│ │ │ ├── files/
│ │ │ │ └── k8s-cluster-critical-pc.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ ├── namespace.j2
│ │ │ ├── node-crb.yml.j2
│ │ │ └── vsphere-rbac.yml.j2
│ │ ├── common_crds/
│ │ │ ├── gateway_api/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ └── tasks/
│ │ │ │ └── main.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ └── prometheus_operator_crds/
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── container_engine_accelerator/
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ └── nvidia_gpu/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ ├── templates/
│ │ │ │ ├── k8s-device-plugin-nvidia-daemonset.yml.j2
│ │ │ │ └── nvidia-driver-install-daemonset.yml.j2
│ │ │ └── vars/
│ │ │ ├── ubuntu-16.yml
│ │ │ └── ubuntu-18.yml
│ │ ├── container_runtimes/
│ │ │ ├── crun/
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yaml
│ │ │ │ └── templates/
│ │ │ │ └── runtimeclass-crun.yml
│ │ │ ├── gvisor/
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yaml
│ │ │ │ └── templates/
│ │ │ │ └── runtimeclass-gvisor.yml.j2
│ │ │ ├── kata_containers/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yaml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yaml
│ │ │ │ └── templates/
│ │ │ │ └── runtimeclass-kata-qemu.yml.j2
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ └── youki/
│ │ │ ├── tasks/
│ │ │ │ └── main.yaml
│ │ │ └── templates/
│ │ │ └── runtimeclass-youki.yml
│ │ ├── csi_driver/
│ │ │ ├── aws_ebs/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── aws-ebs-csi-controllerservice-rbac.yml.j2
│ │ │ │ ├── aws-ebs-csi-controllerservice.yml.j2
│ │ │ │ ├── aws-ebs-csi-driver.yml.j2
│ │ │ │ └── aws-ebs-csi-nodeservice.yml.j2
│ │ │ ├── azuredisk/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ ├── azure-credential-check.yml
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── azure-csi-azuredisk-controller-rbac.yml.j2
│ │ │ │ ├── azure-csi-azuredisk-controller.yml.j2
│ │ │ │ ├── azure-csi-azuredisk-driver.yml.j2
│ │ │ │ ├── azure-csi-azuredisk-node-rbac.yml.j2
│ │ │ │ ├── azure-csi-azuredisk-node.yml.j2
│ │ │ │ ├── azure-csi-cloud-config-secret.yml.j2
│ │ │ │ └── azure-csi-cloud-config.j2
│ │ │ ├── cinder/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ ├── cinder-credential-check.yml
│ │ │ │ │ ├── cinder-write-cacert.yml
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── cinder-csi-cloud-config-secret.yml.j2
│ │ │ │ ├── cinder-csi-cloud-config.j2
│ │ │ │ ├── cinder-csi-controllerplugin-rbac.yml.j2
│ │ │ │ ├── cinder-csi-controllerplugin.yml.j2
│ │ │ │ ├── cinder-csi-driver.yml.j2
│ │ │ │ ├── cinder-csi-nodeplugin-rbac.yml.j2
│ │ │ │ ├── cinder-csi-nodeplugin.yml.j2
│ │ │ │ └── cinder-csi-poddisruptionbudget.yml.j2
│ │ │ ├── csi_crd/
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── volumegroupsnapshotclasses.yml.j2
│ │ │ │ ├── volumegroupsnapshotcontents.yml.j2
│ │ │ │ ├── volumegroupsnapshots.yml.j2
│ │ │ │ ├── volumesnapshotclasses.yml.j2
│ │ │ │ ├── volumesnapshotcontents.yml.j2
│ │ │ │ └── volumesnapshots.yml.j2
│ │ │ ├── gcp_pd/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── gcp-pd-csi-controller.yml.j2
│ │ │ │ ├── gcp-pd-csi-cred-secret.yml.j2
│ │ │ │ ├── gcp-pd-csi-node.yml.j2
│ │ │ │ ├── gcp-pd-csi-sc-regional.yml.j2
│ │ │ │ ├── gcp-pd-csi-sc-zonal.yml.j2
│ │ │ │ └── gcp-pd-csi-setup.yml.j2
│ │ │ ├── upcloud/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── upcloud-csi-controller.yml.j2
│ │ │ │ ├── upcloud-csi-cred-secret.yml.j2
│ │ │ │ ├── upcloud-csi-driver.yml.j2
│ │ │ │ ├── upcloud-csi-node.yml.j2
│ │ │ │ └── upcloud-csi-setup.yml.j2
│ │ │ └── vsphere/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ ├── main.yml
│ │ │ │ └── vsphere-credentials-check.yml
│ │ │ └── templates/
│ │ │ ├── vsphere-csi-cloud-config.j2
│ │ │ ├── vsphere-csi-controller-config.yml.j2
│ │ │ ├── vsphere-csi-controller-deployment.yml.j2
│ │ │ ├── vsphere-csi-controller-rbac.yml.j2
│ │ │ ├── vsphere-csi-controller-service.yml.j2
│ │ │ ├── vsphere-csi-driver.yml.j2
│ │ │ ├── vsphere-csi-namespace.yml.j2
│ │ │ ├── vsphere-csi-node-rbac.yml.j2
│ │ │ └── vsphere-csi-node.yml.j2
│ │ ├── external_cloud_controller/
│ │ │ ├── hcloud/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── external-hcloud-cloud-controller-manager-ds-with-networks.yml.j2
│ │ │ │ ├── external-hcloud-cloud-controller-manager-ds.yml.j2
│ │ │ │ ├── external-hcloud-cloud-role-bindings.yml.j2
│ │ │ │ ├── external-hcloud-cloud-secret.yml.j2
│ │ │ │ └── external-hcloud-cloud-service-account.yml.j2
│ │ │ ├── huaweicloud/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ ├── huaweicloud-credential-check.yml
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── external-huawei-cloud-config-secret.yml.j2
│ │ │ │ ├── external-huawei-cloud-config.j2
│ │ │ │ ├── external-huawei-cloud-controller-manager-ds.yml.j2
│ │ │ │ ├── external-huawei-cloud-controller-manager-role-bindings.yml.j2
│ │ │ │ └── external-huawei-cloud-controller-manager-roles.yml.j2
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── oci/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── external-oci-cloud-config-secret.yml.j2
│ │ │ │ ├── external-oci-cloud-config.yml.j2
│ │ │ │ ├── external-oci-cloud-controller-manager-rbac.yml.j2
│ │ │ │ └── external-oci-cloud-controller-manager.yml.j2
│ │ │ ├── openstack/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ ├── main.yml
│ │ │ │ │ └── openstack-credential-check.yml
│ │ │ │ └── templates/
│ │ │ │ ├── external-openstack-cloud-config-secret.yml.j2
│ │ │ │ ├── external-openstack-cloud-config.j2
│ │ │ │ ├── external-openstack-cloud-controller-manager-ds.yml.j2
│ │ │ │ ├── external-openstack-cloud-controller-manager-role-bindings.yml.j2
│ │ │ │ └── external-openstack-cloud-controller-manager-roles.yml.j2
│ │ │ └── vsphere/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ ├── main.yml
│ │ │ │ └── vsphere-credentials-check.yml
│ │ │ └── templates/
│ │ │ ├── external-vsphere-cloud-controller-manager-ds.yml.j2
│ │ │ ├── external-vsphere-cloud-controller-manager-role-bindings.yml.j2
│ │ │ ├── external-vsphere-cloud-controller-manager-roles.yml.j2
│ │ │ ├── external-vsphere-cpi-cloud-config-secret.yml.j2
│ │ │ └── external-vsphere-cpi-cloud-config.j2
│ │ ├── external_provisioner/
│ │ │ ├── local_path_provisioner/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── local-path-storage-clusterrolebinding.yml.j2
│ │ │ │ ├── local-path-storage-cm.yml.j2
│ │ │ │ ├── local-path-storage-cr.yml.j2
│ │ │ │ ├── local-path-storage-deployment.yml.j2
│ │ │ │ ├── local-path-storage-ns.yml.j2
│ │ │ │ ├── local-path-storage-sa.yml.j2
│ │ │ │ └── local-path-storage-sc.yml.j2
│ │ │ ├── local_volume_provisioner/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ ├── basedirs.yml
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── local-volume-provisioner-clusterrole.yml.j2
│ │ │ │ ├── local-volume-provisioner-clusterrolebinding.yml.j2
│ │ │ │ ├── local-volume-provisioner-cm.yml.j2
│ │ │ │ ├── local-volume-provisioner-ds.yml.j2
│ │ │ │ ├── local-volume-provisioner-ns.yml.j2
│ │ │ │ ├── local-volume-provisioner-sa.yml.j2
│ │ │ │ └── local-volume-provisioner-sc.yml.j2
│ │ │ └── meta/
│ │ │ └── main.yml
│ │ ├── helm/
│ │ │ ├── .gitkeep
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ ├── main.yml
│ │ │ │ └── pyyaml-flatcar.yml
│ │ │ └── vars/
│ │ │ ├── amazon.yml
│ │ │ ├── centos-7.yml
│ │ │ ├── centos.yml
│ │ │ ├── debian.yml
│ │ │ ├── fedora.yml
│ │ │ ├── redhat-7.yml
│ │ │ ├── redhat.yml
│ │ │ ├── suse.yml
│ │ │ └── ubuntu.yml
│ │ ├── ingress_controller/
│ │ │ ├── alb_ingress_controller/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── alb-ingress-clusterrole.yml.j2
│ │ │ │ ├── alb-ingress-clusterrolebinding.yml.j2
│ │ │ │ ├── alb-ingress-deploy.yml.j2
│ │ │ │ ├── alb-ingress-ns.yml.j2
│ │ │ │ └── alb-ingress-sa.yml.j2
│ │ │ ├── cert_manager/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── cert-manager.crds.yml.j2
│ │ │ │ └── cert-manager.yml.j2
│ │ │ └── meta/
│ │ │ └── main.yml
│ │ ├── kubelet-csr-approver/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ └── meta/
│ │ │ └── main.yml
│ │ ├── meta/
│ │ │ └── main.yml
│ │ ├── metallb/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ ├── layer2.yaml.j2
│ │ │ ├── layer3.yaml.j2
│ │ │ ├── metallb.yaml.j2
│ │ │ └── pools.yaml.j2
│ │ ├── metrics_server/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ ├── auth-delegator.yaml.j2
│ │ │ ├── auth-reader.yaml.j2
│ │ │ ├── metrics-apiservice.yaml.j2
│ │ │ ├── metrics-server-deployment.yaml.j2
│ │ │ ├── metrics-server-sa.yaml.j2
│ │ │ ├── metrics-server-service.yaml.j2
│ │ │ ├── resource-reader-clusterrolebinding.yaml.j2
│ │ │ └── resource-reader.yaml.j2
│ │ ├── node_feature_discovery/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ ├── nfd-api-crds.yaml.j2
│ │ │ ├── nfd-clusterrole.yaml.j2
│ │ │ ├── nfd-clusterrolebinding.yaml.j2
│ │ │ ├── nfd-gc.yaml.j2
│ │ │ ├── nfd-master-conf.yaml.j2
│ │ │ ├── nfd-master.yaml.j2
│ │ │ ├── nfd-ns.yaml.j2
│ │ │ ├── nfd-role.yaml.j2
│ │ │ ├── nfd-rolebinding.yaml.j2
│ │ │ ├── nfd-service.yaml.j2
│ │ │ ├── nfd-serviceaccount.yaml.j2
│ │ │ ├── nfd-topologyupdater-conf.yaml.j2
│ │ │ ├── nfd-worker-conf.yaml.j2
│ │ │ └── nfd-worker.yaml.j2
│ │ ├── persistent_volumes/
│ │ │ ├── aws-ebs-csi/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ └── aws-ebs-csi-storage-class.yml.j2
│ │ │ ├── azuredisk-csi/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ └── azure-csi-storage-class.yml.j2
│ │ │ ├── cinder-csi/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ └── cinder-csi-storage-class.yml.j2
│ │ │ ├── gcp-pd-csi/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ └── gcp-pd-csi-storage-class.yml.j2
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ └── upcloud-csi/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ └── upcloud-csi-storage-class.yml.j2
│ │ ├── policy_controller/
│ │ │ ├── calico/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ ├── calico-kube-controllers.yml.j2
│ │ │ │ ├── calico-kube-cr.yml.j2
│ │ │ │ ├── calico-kube-crb.yml.j2
│ │ │ │ └── calico-kube-sa.yml.j2
│ │ │ └── meta/
│ │ │ └── main.yml
│ │ ├── registry/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ ├── registry-cm.yml.j2
│ │ │ ├── registry-ing.yml.j2
│ │ │ ├── registry-ns.yml.j2
│ │ │ ├── registry-pvc.yml.j2
│ │ │ ├── registry-rs.yml.j2
│ │ │ ├── registry-sa.yml.j2
│ │ │ ├── registry-secrets.yml.j2
│ │ │ └── registry-svc.yml.j2
│ │ ├── scheduler_plugins/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ ├── appgroup.diktyo.x-k8s.io_appgroups.yaml.j2
│ │ │ ├── cm-scheduler-plugins.yaml.j2
│ │ │ ├── deploy-scheduler-plugins.yaml.j2
│ │ │ ├── namespace.yaml.j2
│ │ │ ├── networktopology.diktyo.x-k8s.io_networktopologies.yaml.j2
│ │ │ ├── rbac-scheduler-plugins.yaml.j2
│ │ │ ├── sa-scheduler-plugins.yaml.j2
│ │ │ ├── scheduling.x-k8s.io_elasticquotas.yaml.j2
│ │ │ ├── scheduling.x-k8s.io_podgroups.yaml.j2
│ │ │ └── topology.node.k8s.io_noderesourcetopologies.yaml.j2
│ │ ├── snapshots/
│ │ │ ├── cinder-csi/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ ├── tasks/
│ │ │ │ │ └── main.yml
│ │ │ │ └── templates/
│ │ │ │ └── cinder-csi-snapshot-class.yml.j2
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ └── snapshot-controller/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ ├── rbac-snapshot-controller.yml.j2
│ │ │ ├── snapshot-controller.yml.j2
│ │ │ └── snapshot-ns.yml.j2
│ │ └── utils/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ └── vars/
│ │ └── main.yml
│ ├── kubespray-defaults/
│ │ └── tasks/
│ │ └── main.yml
│ ├── kubespray_defaults/
│ │ ├── defaults/
│ │ │ └── main/
│ │ │ ├── download.yml
│ │ │ └── main.yml
│ │ └── vars/
│ │ └── main/
│ │ ├── checksums.yml
│ │ └── main.yml
│ ├── network_facts/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ ├── meta/
│ │ │ └── main.yml
│ │ └── tasks/
│ │ └── main.yaml
│ ├── network_plugin/
│ │ ├── calico/
│ │ │ ├── files/
│ │ │ │ └── openssl.conf
│ │ │ ├── handlers/
│ │ │ │ └── main.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── rr/
│ │ │ │ ├── defaults/
│ │ │ │ │ └── main.yml
│ │ │ │ └── tasks/
│ │ │ │ ├── main.yml
│ │ │ │ ├── pre.yml
│ │ │ │ └── update-node.yml
│ │ │ ├── tasks/
│ │ │ │ ├── calico_apiserver_certs.yml
│ │ │ │ ├── check.yml
│ │ │ │ ├── install.yml
│ │ │ │ ├── main.yml
│ │ │ │ ├── peer_with_calico_rr.yml
│ │ │ │ ├── peer_with_router.yml
│ │ │ │ ├── pre.yml
│ │ │ │ ├── repos.yml
│ │ │ │ ├── reset.yml
│ │ │ │ └── typha_certs.yml
│ │ │ ├── templates/
│ │ │ │ ├── calico-apiserver-ns.yml.j2
│ │ │ │ ├── calico-apiserver.yml.j2
│ │ │ │ ├── calico-config.yml.j2
│ │ │ │ ├── calico-cr.yml.j2
│ │ │ │ ├── calico-crb.yml.j2
│ │ │ │ ├── calico-ipamconfig.yml.j2
│ │ │ │ ├── calico-node-sa.yml.j2
│ │ │ │ ├── calico-node.yml.j2
│ │ │ │ ├── calico-typha.yml.j2
│ │ │ │ ├── calicoctl.etcd.sh.j2
│ │ │ │ ├── calicoctl.kdd.sh.j2
│ │ │ │ ├── kubernetes-services-endpoint.yml.j2
│ │ │ │ └── make-ssl-calico.sh.j2
│ │ │ └── vars/
│ │ │ ├── amazon.yml
│ │ │ ├── centos-9.yml
│ │ │ ├── debian.yml
│ │ │ ├── fedora.yml
│ │ │ ├── opensuse.yml
│ │ │ ├── redhat-9.yml
│ │ │ ├── redhat.yml
│ │ │ └── rocky-9.yml
│ │ ├── calico_defaults/
│ │ │ └── defaults/
│ │ │ └── main.yml
│ │ ├── cilium/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ ├── apply.yml
│ │ │ │ ├── check.yml
│ │ │ │ ├── install.yml
│ │ │ │ ├── main.yml
│ │ │ │ ├── reset.yml
│ │ │ │ └── reset_iface.yml
│ │ │ └── templates/
│ │ │ ├── cilium/
│ │ │ │ ├── cilium-bgp-advertisement.yml.j2
│ │ │ │ ├── cilium-bgp-cluster-config.yml.j2
│ │ │ │ ├── cilium-bgp-node-config-override.yml.j2
│ │ │ │ ├── cilium-bgp-peer-config.yml.j2
│ │ │ │ ├── cilium-bgp-peering-policy.yml.j2
│ │ │ │ └── cilium-loadbalancer-ip-pool.yml.j2
│ │ │ └── values.yaml.j2
│ │ ├── cni/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── custom_cni/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── flannel/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ ├── main.yml
│ │ │ │ └── reset.yml
│ │ │ └── templates/
│ │ │ ├── cni-flannel-rbac.yml.j2
│ │ │ └── cni-flannel.yml.j2
│ │ ├── kube-ovn/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ ├── cni-kube-ovn-crd.yml.j2
│ │ │ ├── cni-kube-ovn.yml.j2
│ │ │ └── cni-ovn.yml.j2
│ │ ├── kube-router/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── handlers/
│ │ │ │ └── main.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ ├── annotate.yml
│ │ │ │ ├── main.yml
│ │ │ │ └── reset.yml
│ │ │ └── templates/
│ │ │ ├── cni-conf.json.j2
│ │ │ ├── kube-router.yml.j2
│ │ │ └── kubeconfig.yml.j2
│ │ ├── macvlan/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── files/
│ │ │ │ ├── ifdown-local
│ │ │ │ ├── ifdown-macvlan
│ │ │ │ ├── ifup-local
│ │ │ │ └── ifup-macvlan
│ │ │ ├── handlers/
│ │ │ │ └── main.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ ├── 10-macvlan.conf.j2
│ │ │ ├── 99-loopback.conf.j2
│ │ │ ├── centos-network-macvlan.cfg.j2
│ │ │ ├── centos-postdown-macvlan.cfg.j2
│ │ │ ├── centos-postup-macvlan.cfg.j2
│ │ │ ├── centos-routes-macvlan.cfg.j2
│ │ │ ├── coreos-device-macvlan.cfg.j2
│ │ │ ├── coreos-interface-macvlan.cfg.j2
│ │ │ ├── coreos-network-macvlan.cfg.j2
│ │ │ ├── coreos-service-nat_ouside.j2
│ │ │ └── debian-network-macvlan.cfg.j2
│ │ ├── multus/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ ├── files/
│ │ │ │ ├── multus-clusterrole.yml
│ │ │ │ ├── multus-clusterrolebinding.yml
│ │ │ │ ├── multus-crd.yml
│ │ │ │ └── multus-serviceaccount.yml
│ │ │ ├── meta/
│ │ │ │ └── main.yml
│ │ │ ├── tasks/
│ │ │ │ └── main.yml
│ │ │ └── templates/
│ │ │ └── multus-daemonset.yml.j2
│ │ ├── ovn4nfv/
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ └── tasks/
│ │ └── main.yml
│ ├── recover_control_plane/
│ │ ├── control-plane/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── etcd/
│ │ │ └── tasks/
│ │ │ ├── main.yml
│ │ │ └── recover_lost_quorum.yml
│ │ └── post-recover/
│ │ └── tasks/
│ │ └── main.yml
│ ├── remove-node/
│ │ ├── post-remove/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── pre-remove/
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ └── remove-etcd-node/
│ │ └── tasks/
│ │ └── main.yml
│ ├── remove_node/
│ │ └── pre_remove/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ └── tasks/
│ │ └── main.yml
│ ├── reset/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ └── tasks/
│ │ └── main.yml
│ ├── system_packages/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ ├── tasks/
│ │ │ └── main.yml
│ │ └── vars/
│ │ └── main.yml
│ ├── upgrade/
│ │ ├── post-upgrade/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ ├── pre-upgrade/
│ │ │ ├── defaults/
│ │ │ │ └── main.yml
│ │ │ └── tasks/
│ │ │ └── main.yml
│ │ └── system-upgrade/
│ │ └── tasks/
│ │ ├── apt.yml
│ │ ├── main.yml
│ │ └── yum.yml
│ ├── validate_inventory/
│ │ ├── meta/
│ │ │ └── main.yml
│ │ └── tasks/
│ │ └── main.yml
│ └── win_nodes/
│ └── kubernetes_patch/
│ ├── defaults/
│ │ └── main.yml
│ └── tasks/
│ └── main.yml
├── scale.yml
├── scripts/
│ ├── Dockerfile.j2
│ ├── assert-sorted-checksums.yml
│ ├── collect-info.yaml
│ ├── component_hash_update/
│ │ ├── pyproject.toml
│ │ └── src/
│ │ └── component_hash_update/
│ │ ├── __init__.py
│ │ ├── components.py
│ │ ├── download.py
│ │ └── list_releases.graphql
│ ├── galaxy_version.py
│ ├── gen_docs_sidebar.sh
│ ├── get_node_ids.sh
│ ├── gitlab-runner.sh
│ ├── openstack-cleanup/
│ │ ├── .gitignore
│ │ ├── README.md
│ │ ├── main.py
│ │ └── requirements.txt
│ ├── pipeline.Dockerfile.j2
│ ├── propagate_ansible_variables.yml
│ └── readme_versions.md.j2
├── test-infra/
│ ├── image-builder/
│ │ ├── Makefile
│ │ ├── OWNERS
│ │ ├── README.md
│ │ ├── cluster.yml
│ │ ├── hosts.ini
│ │ └── roles/
│ │ └── kubevirt-images/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ ├── tasks/
│ │ │ └── main.yml
│ │ └── templates/
│ │ └── Dockerfile
│ └── vagrant-docker/
│ ├── Dockerfile
│ ├── README.md
│ └── build.sh
├── tests/
│ ├── Makefile
│ ├── ansible.cfg
│ ├── cloud_playbooks/
│ │ ├── create-kubevirt.yml
│ │ └── roles/
│ │ └── packet-ci/
│ │ ├── defaults/
│ │ │ └── main.yml
│ │ ├── tasks/
│ │ │ └── main.yml
│ │ ├── templates/
│ │ │ └── vm.yml.j2
│ │ └── vars/
│ │ └── main.yml
│ ├── common_vars.yml
│ ├── files/
│ │ ├── almalinux9-calico-ha-ebpf.yml
│ │ ├── almalinux9-calico-nodelocaldns-secondary.yml
│ │ ├── almalinux9-calico-remove-node
│ │ ├── almalinux9-calico-remove-node.yml
│ │ ├── almalinux9-calico.yml
│ │ ├── almalinux9-crio.yml
│ │ ├── almalinux9-docker.yml
│ │ ├── almalinux9-kube-ovn.yml
│ │ ├── amazon-linux-2-all-in-one.yml
│ │ ├── custom_cni/
│ │ │ ├── README.md
│ │ │ ├── cilium.yaml
│ │ │ └── values.yaml
│ │ ├── debian11-calico-collection.yml
│ │ ├── debian11-calico-upgrade
│ │ ├── debian11-calico-upgrade-once
│ │ ├── debian11-calico-upgrade-once.yml
│ │ ├── debian11-calico-upgrade.yml
│ │ ├── debian11-custom-cni.yml
│ │ ├── debian11-docker.yml
│ │ ├── debian11-kubelet-csr-approver.yml
│ │ ├── debian11-macvlan.yml
│ │ ├── debian12-calico.yml
│ │ ├── debian12-cilium-svc-proxy.yml
│ │ ├── debian12-cilium.yml
│ │ ├── debian12-custom-cni-helm.yml
│ │ ├── debian12-docker.yml
│ │ ├── debian13-calico.yml
│ │ ├── debian13-cilium.yml
│ │ ├── fedora39-calico-selinux.yml
│ │ ├── fedora39-calico-swap-selinux.yml
│ │ ├── fedora39-crio.yml
│ │ ├── fedora39-kube-router.yml
│ │ ├── fedora40-docker-calico.yml
│ │ ├── fedora40-docker.calico
│ │ ├── fedora40-flannel-crio-collection-scale.yml
│ │ ├── fedora41-calico-selinux.yml
│ │ ├── fedora41-calico-swap-selinux.yml
│ │ ├── fedora41-crio.yml
│ │ ├── fedora41-kube-router.yml
│ │ ├── fedora42-calico.yml
│ │ ├── flatcar4081-calico.yml
│ │ ├── openeuler24-calico.yml
│ │ ├── rockylinux10-calico.yml
│ │ ├── rockylinux10-cilium
│ │ ├── rockylinux10-cilium.yml
│ │ ├── rockylinux9-calico.yml
│ │ ├── rockylinux9-cilium
│ │ ├── rockylinux9-cilium.yml
│ │ ├── tf-elastx_ubuntu24-calico.yml
│ │ ├── ubuntu22-all-in-one-docker.yml
│ │ ├── ubuntu22-calico-all-in-one-upgrade
│ │ ├── ubuntu22-calico-all-in-one-upgrade.yml
│ │ ├── ubuntu22-calico-all-in-one.yml
│ │ ├── ubuntu22-crio.yml
│ │ ├── ubuntu24-all-in-one-docker.yml
│ │ ├── ubuntu24-calico-all-in-one
│ │ ├── ubuntu24-calico-all-in-one-hardening.yml
│ │ ├── ubuntu24-calico-all-in-one.yml
│ │ ├── ubuntu24-calico-dual-stack.rb
│ │ ├── ubuntu24-calico-dual-stack.yml
│ │ ├── ubuntu24-calico-etcd-datastore.yml
│ │ ├── ubuntu24-calico-etcd-kubeadm-upgrade-ha
│ │ ├── ubuntu24-calico-etcd-kubeadm-upgrade-ha.yml
│ │ ├── ubuntu24-calico-etcd-kubeadm.yml
│ │ ├── ubuntu24-calico-ha-recover
│ │ ├── ubuntu24-calico-ha-recover-noquorum
│ │ ├── ubuntu24-calico-ha-recover-noquorum.yml
│ │ ├── ubuntu24-calico-ha-recover.yml
│ │ ├── ubuntu24-calico-ha-wireguard.yml
│ │ ├── ubuntu24-calico-ipv6only-stack.rb
│ │ ├── ubuntu24-calico-ipv6only-stack.yml
│ │ ├── ubuntu24-cilium-sep.yml
│ │ ├── ubuntu24-crio-scale.yml
│ │ ├── ubuntu24-crio-upgrade
│ │ ├── ubuntu24-crio-upgrade.yml
│ │ ├── ubuntu24-flannel-collection.yml
│ │ ├── ubuntu24-flannel-ha-once.yml
│ │ ├── ubuntu24-flannel-ha.yml
│ │ ├── ubuntu24-flannel.yml
│ │ ├── ubuntu24-ha-separate-etcd
│ │ ├── ubuntu24-ha-separate-etcd.yml
│ │ ├── ubuntu24-kube-router-sep.yml
│ │ └── ubuntu24-kube-router-svc-proxy.yml
│ ├── requirements.txt
│ ├── scripts/
│ │ ├── check-templates.py
│ │ ├── collection-build-install.sh
│ │ ├── md-table/
│ │ │ ├── main.py
│ │ │ └── table.md.j2
│ │ ├── molecule_run.sh
│ │ ├── opentofu_install.sh
│ │ ├── rebase.sh
│ │ ├── testcases_run.sh
│ │ ├── vagrant-install.sh
│ │ ├── vagrant-validate.sh
│ │ └── vagrant_clean.sh
│ └── testcases/
│ ├── 000_install-hydrophone.yml
│ ├── 010_check-apiserver.yml
│ ├── 015_check-nodes-ready.yml
│ ├── 020_check-pods-running.yml
│ ├── 025_check-csr-request.yml
│ ├── 030_check-network.yml
│ ├── 040_check-network-adv.yml
│ ├── 100_check-k8s-conformance.yml
│ ├── roles/
│ │ └── cluster-dump/
│ │ └── tasks/
│ │ └── main.yml
│ └── tests.yml
├── upgrade-cluster.yml
└── upgrade_cluster.yml
================================================
FILE CONTENTS
================================================
================================================
FILE: .ansible-lint
================================================
---
skip_list:
# see https://docs.ansible.com/ansible-lint/rules/default_rules.html for a list of all default rules
# DO NOT add any other rules to this skip_list, instead use local `# noqa` with a comment explaining WHY it is necessary
# These rules are intentionally skipped:
#
# [role-name] "meta/main.yml" Role name role-name does not match ``^+$`` pattern
# Meta roles in Kubespray don't need proper names
# (Disabled in June 2021)
- 'role-name'
# [var-naming]
# In Kubespray we use variables that use camelCase to match their k8s counterparts
# (Disabled in June 2021)
- 'var-naming[pattern]'
# Variables names from within roles in kubespray don't need role name as a prefix
- 'var-naming[no-role-prefix]'
# [fqcn-builtins]
# Roles in kubespray don't need fully qualified collection names
# (Disabled in Feb 2023)
- 'fqcn-builtins'
# We use template in names
- 'name[template]'
# No changed-when on commands
# (Disabled in June 2023 after ansible upgrade; FIXME)
- 'no-changed-when'
# Disable run-once check with free strategy
# (Disabled in June 2023 after ansible upgrade; FIXME)
- 'run-once[task]'
- 'jinja[spacing]'
exclude_paths:
# Generated files
- tests/files/custom_cni/cilium.yaml
- venv
- .github
- .ansible
- .cache
- .gitlab-ci.yml
- .gitlab-ci
mock_modules:
- gluster.gluster.gluster_volume
================================================
FILE: .ansible-lint-ignore
================================================
# This file contains ignores rule violations for ansible-lint
inventory/sample/group_vars/k8s_cluster/k8s-cluster.yml jinja[spacing]
roles/kubernetes/control-plane/defaults/main/kube-proxy.yml jinja[spacing]
roles/kubernetes/control-plane/defaults/main/main.yml jinja[spacing]
roles/kubernetes/kubeadm/defaults/main.yml jinja[spacing]
roles/kubernetes/node/defaults/main.yml jinja[spacing]
roles/kubernetes/preinstall/defaults/main.yml jinja[spacing]
roles/kubespray-defaults/defaults/main/main.yml jinja[spacing]
================================================
FILE: .editorconfig
================================================
root = true
[*.{yaml,yml,yml.j2,yaml.j2}]
indent_style = space
indent_size = 2
trim_trailing_whitespace = true
insert_final_newline = true
charset = utf-8
[{Dockerfile}]
indent_style = space
indent_size = 2
trim_trailing_whitespace = true
insert_final_newline = true
charset = utf-8
================================================
FILE: .gitattributes
================================================
docs/_sidebar.md linguist-generated=true
================================================
FILE: .github/ISSUE_TEMPLATE/bug-report.yaml
================================================
---
name: Bug Report
description: Report a bug encountered while using Kubespray
labels: kind/bug
body:
- type: markdown
attributes:
value: |
Please, be ready for followup questions, and please respond in a timely
manner. If we can't reproduce a bug or think a feature already exists, we
might close your issue. If we're wrong, PLEASE feel free to reopen it and
explain why.
- type: textarea
id: problem
attributes:
label: What happened?
description: |
Please provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner.
validations:
required: true
- type: textarea
id: expected
attributes:
label: What did you expect to happen?
validations:
required: true
- type: textarea
id: repro
attributes:
label: How can we reproduce it (as minimally and precisely as possible)?
validations:
required: true
- type: markdown
attributes:
value: '### Environment'
- type: dropdown
id: os
attributes:
label: OS
options:
- 'RHEL 9'
- 'RHEL 8'
- 'Fedora 40'
- 'Ubuntu 24'
- 'Ubuntu 22'
- 'Ubuntu 20'
- 'Debian 12'
- 'Debian 11'
- 'Flatcar Container Linux'
- 'openSUSE Leap'
- 'openSUSE Tumbleweed'
- 'Oracle Linux 9'
- 'Oracle Linux 8'
- 'AlmaLinux 9'
- 'AlmaLinux 8'
- 'Rocky Linux 9'
- 'Rocky Linux 8'
- 'Amazon Linux 2'
- 'Kylin Linux Advanced Server V10'
- 'UOS Linux 20'
- 'openEuler 24'
- 'openEuler 22'
- 'openEuler 20'
- 'Other|Unsupported'
validations:
required: true
- type: textarea
id: ansible_version
attributes:
label: Version of Ansible
placeholder: 'ansible --version'
validations:
required: true
- type: input
id: python_version
attributes:
label: Version of Python
placeholder: 'python --version'
validations:
required: true
- type: input
id: kubespray_version
attributes:
label: Version of Kubespray (commit)
placeholder: 'git rev-parse --short HEAD'
validations:
required: true
- type: dropdown
id: network_plugin
attributes:
label: Network plugin used
options:
- calico
- cilium
- cni
- custom_cni
- flannel
- kube-ovn
- kube-router
- macvlan
- meta
- multus
- ovn4nfv
validations:
required: true
- type: textarea
id: inventory
attributes:
label: Full inventory with variables
placeholder: 'ansible -i inventory/sample/inventory.ini all -m debug -a "var=hostvars[inventory_hostname]"'
description: We recommend using snippets services like https://gist.github.com/ etc.
validations:
required: true
- type: input
id: ansible_command
attributes:
label: Command used to invoke ansible
validations:
required: true
- type: textarea
id: ansible_output
attributes:
label: Output of ansible run
description: We recommend using snippets services like https://gist.github.com/ etc.
validations:
required: true
- type: textarea
id: anything_else
attributes:
label: Anything else we need to know
description: |
By running scripts/collect-info.yaml you can get a lot of useful informations.
Script can be started by:
ansible-playbook -i <inventory_file_path> -u <ssh_user> -e ansible_ssh_user=<ssh_user> -b --become-user=root -e dir=`pwd` scripts/collect-info.yaml
(If you using CoreOS remember to add '-e ansible_python_interpreter=/opt/bin/python').
After running this command you can find logs in `pwd`/logs.tar.gz. You can even upload somewhere entire file and paste link here
================================================
FILE: .github/ISSUE_TEMPLATE/config.yml
================================================
---
blank_issues_enabled: false
contact_links:
- name: Support Request
url: https://kubernetes.slack.com/channels/kubespray
about: Support request or question relating to Kubernetes
================================================
FILE: .github/ISSUE_TEMPLATE/enhancement.yaml
================================================
---
name: Enhancement Request
description: Suggest an enhancement to the Kubespray project
labels: kind/feature
body:
- type: markdown
attributes:
value: Please only use this template for submitting enhancement requests
- type: textarea
id: what
attributes:
label: What would you like to be added
validations:
required: true
- type: textarea
id: why
attributes:
label: Why is this needed
validations:
required: true
================================================
FILE: .github/ISSUE_TEMPLATE/failing-test.yaml
================================================
---
name: Failing Test
description: Report test failures in Kubespray CI jobs
labels: kind/failing-test
body:
- type: markdown
attributes:
value: Please only use this template for submitting reports about failing tests in Kubespray CI jobs
- type: textarea
id: failing_jobs
attributes:
label: Which jobs are failing ?
validations:
required: true
- type: textarea
id: failing_tests
attributes:
label: Which tests are failing ?
validations:
required: true
- type: input
id: since_when
attributes:
label: Since when has it been failing ?
validations:
required: true
- type: textarea
id: failure_reason
attributes:
label: Reason for failure
description: If you don't know and have no guess, just put "Unknown"
validations:
required: true
- type: textarea
id: anything_else
attributes:
label: Anything else we need to know
================================================
FILE: .github/PULL_REQUEST_TEMPLATE.md
================================================
<!-- Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md and developer guide https://git.k8s.io/community/contributors/devel/development.md
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. Follow the instructions for writing a release note: https://git.k8s.io/community/contributors/guide/release-notes.md
6. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->
**What type of PR is this?**
> Uncomment only one ` /kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line:
>
> /kind api-change
> /kind bug
> /kind cleanup
> /kind design
> /kind documentation
> /kind failing-test
> /kind feature
> /kind flake
**What this PR does / why we need it**:
**Which issue(s) this PR fixes**:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #
**Special notes for your reviewer**:
**Does this PR introduce a user-facing change?**:
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".
-->
```release-note
```
================================================
FILE: .github/dependabot.yml
================================================
version: 2
updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
labels:
- dependencies
- release-note-none
groups:
molecule:
patterns:
- molecule
- molecule-plugins*
- package-ecosystem: "github-actions"
directory: "/"
labels:
- release-note-none
- ci-short
schedule:
interval: "weekly"
================================================
FILE: .github/workflows/auto-label-os.yml
================================================
name: Issue labeler
on:
issues:
types: [opened]
permissions:
contents: read
jobs:
label-component:
runs-on: ubuntu-latest
permissions:
issues: write
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
- name: Parse issue form
uses: stefanbuck/github-issue-parser@10dcc54158ba4c137713d9d69d70a2da63b6bda3
id: issue-parser
with:
template-path: .github/ISSUE_TEMPLATE/bug-report.yaml
- name: Set labels based on OS field
uses: redhat-plumbers-in-action/advanced-issue-labeler@b80ae64e3e156e9c111b075bfa04b295d54e8e2e
with:
issue-form: ${{ steps.issue-parser.outputs.jsonString }}
section: os
block-list: |
None
Other
token: ${{ secrets.GITHUB_TOKEN }}
================================================
FILE: .github/workflows/upgrade-patch-versions-schedule.yml
================================================
name: Upgrade Kubespray components with new patches versions - all branches
on:
schedule:
- cron: '22 2 * * *' # every day, 02:22 UTC
workflow_dispatch:
permissions: {}
jobs:
get-releases-branches:
if: github.repository == 'kubernetes-sigs/kubespray'
runs-on: ubuntu-latest
outputs:
branches: ${{ steps.get-branches.outputs.data }}
steps:
- uses: octokit/graphql-action@ddde8ebb2493e79f390e6449c725c21663a67505
id: get-branches
with:
query: |
query get_release_branches($owner:String!, $name:String!) {
repository(owner:$owner, name:$name) {
refs(refPrefix: "refs/heads/",
first: 3,
query: "release-",
orderBy: {
field: ALPHABETICAL,
direction: DESC
}) {
nodes {
name
}
}
}
}
variables: |
owner: ${{ github.repository_owner }}
name: ${{ github.event.repository.name }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
update-versions:
needs: get-releases-branches
strategy:
fail-fast: false
matrix:
branch:
- name: ${{ github.event.repository.default_branch }}
- ${{ fromJSON(needs.get-releases-branches.outputs.branches).repository.refs.nodes }}
uses: ./.github/workflows/upgrade-patch-versions.yml
permissions:
contents: write
pull-requests: write
name: Update patch updates on ${{ matrix.branch.name }}
with:
branch: ${{ matrix.branch.name }}
================================================
FILE: .github/workflows/upgrade-patch-versions.yml
================================================
on:
workflow_call:
inputs:
branch:
description: Which branch to update with new patch versions
default: master
required: true
type: string
jobs:
update-patch-versions:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
with:
ref: ${{ inputs.branch }}
- uses: actions/setup-python@v6
with:
python-version: '3.13'
cache: 'pip'
- run: pip install scripts/component_hash_update pre-commit
- run: update-hashes
env:
API_KEY: ${{ secrets.GITHUB_TOKEN }}
- uses: actions/cache@v5
with:
key: pre-commit-hook-propagate
path: |
~/.cache/pre-commit
- run: pre-commit run --all-files propagate-ansible-variables
continue-on-error: true
- uses: peter-evans/create-pull-request@c0f553fe549906ede9cf27b5156039d195d2ece0
with:
commit-message: Patch versions updates
title: Patch versions updates - ${{ inputs.branch }}
labels: bot
branch: component_hash_update/${{ inputs.branch }}
sign-commits: true
body: |
/kind feature
```release-note
NONE
```
================================================
FILE: .gitignore
================================================
.vagrant
*.retry
**/vagrant_ansible_inventory
*.iml
temp
contrib/offline/container-images
contrib/offline/container-images.tar.gz
contrib/offline/offline-files
contrib/offline/offline-files.tar.gz
.idea
.vscode
.tox
.cache
*.bak
*.tfstate
*.tfstate*backup
*.lock.hcl
.terraform/
contrib/terraform/aws/credentials.tfvars
.terraform.lock.hcl
/ssh-bastion.conf
**/*.sw[pon]
*~
vagrant/
plugins/mitogen
# Ansible inventory
inventory/*
!inventory/local
!inventory/sample
inventory/*/artifacts/
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# Distribution / packaging
.Python
env/
build/
credentials/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
.hypothesis/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# IPython Notebook
.ipynb_checkpoints
# pyenv
.python-version
# dotenv
.env
# virtualenv
venv/
ENV/
# molecule
roles/**/molecule/**/__pycache__/
# macOS
.DS_Store
# Temp location used by our scripts
scripts/tmp/
tmp.md
# Ansible collection files
kubernetes_sigs-kubespray*tar.gz
ansible_collections
================================================
FILE: .gitlab-ci/build.yml
================================================
---
pipeline-image:
cache:
key: $CI_COMMIT_REF_SLUG
paths:
- image-cache
tags:
- ffci
stage: build
image: moby/buildkit:rootless
variables:
BUILDKITD_FLAGS: --oci-worker-no-process-sandbox
CACHE_IMAGE: $CI_REGISTRY_IMAGE/pipeline:cache
# TODO: remove the override
# currently rebase.sh depends on bash (not available in the kaniko image)
# once we have a simpler rebase (which should be easy if the target branch ref is available as variable
# we'll be able to rebase here as well hopefully
before_script:
- mkdir -p ~/.docker
- echo "{\"auths\":{\"$CI_REGISTRY\":{\"auth\":\"$(echo -n ${CI_REGISTRY_USER}:${CI_REGISTRY_PASSWORD} | base64)\"}}}" > ~/.docker/config.json
script:
- |
buildctl-daemonless.sh build \
--frontend dockerfile.v0 \
--local context=$CI_PROJECT_DIR \
--local dockerfile=$CI_PROJECT_DIR \
--opt filename=pipeline.Dockerfile \
--export-cache type=registry,ref=$CACHE_IMAGE \
--import-cache type=registry,ref=$CACHE_IMAGE \
--output type=image,name=$PIPELINE_IMAGE,push=true
================================================
FILE: .gitlab-ci/kubevirt.yml
================================================
---
.kubevirt:
extends: .job-moderated
interruptible: true
script:
- ansible-playbook tests/cloud_playbooks/create-kubevirt.yml
-e @"tests/files/${TESTCASE}.yml"
- ./tests/scripts/testcases_run.sh
variables:
ANSIBLE_TIMEOUT: "120"
tags:
- ffci
needs:
- pipeline-image
# TODO: generate testcases matrixes from the files in tests/files/
# this is needed to avoid the need for PR rebasing when a job was added or removed in the target branch
# (currently, a removed job in the target branch breaks the tests, because the
# pipeline definition is parsed by gitlab before the rebase.sh script)
# CI template for PRs
pr:
stage: deploy-part1
rules:
- if: $PR_LABELS =~ /.*ci-short.*/
when: manual
allow_failure: true
- if: $CI_COMMIT_BRANCH =~ /^pr-.*$/
when: on_success
- if: $CI_PIPELINE_SOURCE == "schedule" && $CI_PIPELINE_SCHEDULE_DESCRIPTION == "daily-ci"
when: on_success
- when: manual
allow_failure: true
extends: .kubevirt
parallel:
matrix:
- TESTCASE:
- almalinux9-crio
- almalinux9-kube-ovn
- debian11-calico-collection
- debian11-macvlan
- debian12-cilium
- debian13-cilium
- fedora39-kube-router
- fedora41-kube-router
- fedora42-calico
- rockylinux9-cilium
- rockylinux10-cilium
- ubuntu22-calico-all-in-one
- ubuntu22-calico-all-in-one-upgrade
- ubuntu24-calico-etcd-datastore
- ubuntu24-calico-all-in-one-hardening
- ubuntu24-cilium-sep
- ubuntu24-crio-scale
- ubuntu24-crio-upgrade
- ubuntu24-flannel-collection
- ubuntu24-kube-router-sep
- ubuntu24-kube-router-svc-proxy
- ubuntu24-ha-separate-etcd
- fedora40-flannel-crio-collection-scale
- openeuler24-calico
# This is for flakey test so they don't disrupt the PR worklflow too much.
# Jobs here MUST have a open issue so we don't lose sight of them
pr-flakey:
extends: pr
retry: 1
parallel:
matrix:
- TESTCASE:
- flatcar4081-calico # https://github.com/kubernetes-sigs/kubespray/issues/12309
# The ubuntu24-calico-all-in-one jobs are meant as early stages to prevent running the full CI if something is horribly broken
ubuntu24-calico-all-in-one:
stage: deploy-part1
extends: .kubevirt
variables:
TESTCASE: ubuntu24-calico-all-in-one
rules:
- if: $CI_COMMIT_BRANCH =~ /^pr-.*$/
when: on_success
- if: $CI_PIPELINE_SOURCE == "schedule" && $CI_PIPELINE_SCHEDULE_DESCRIPTION == "daily-ci"
when: on_success
- when: manual
allow_failure: true
pr_full:
extends: .kubevirt
stage: deploy-extended
rules:
- if: $PR_LABELS =~ /.*ci-full.*/
when: on_success
- if: $CI_PIPELINE_SOURCE == "schedule" && $CI_PIPELINE_SCHEDULE_DESCRIPTION == "daily-ci"
when: on_success
# Else run as manual
- when: manual
allow_failure: true
parallel:
matrix:
- TESTCASE:
- almalinux9-calico-ha-ebpf
- almalinux9-calico-nodelocaldns-secondary
- debian11-custom-cni
- debian11-kubelet-csr-approver
- debian12-custom-cni-helm
- fedora39-calico-swap-selinux
- fedora39-crio
- fedora41-calico-swap-selinux
- fedora41-crio
- ubuntu24-calico-ha-wireguard
- ubuntu24-flannel-ha
- ubuntu24-flannel-ha-once
# Need an update of the container image to use schema v2
# update: quay.io/kubespray/vm-amazon-linux-2:latest
manual:
extends: pr_full
parallel:
matrix:
- TESTCASE:
- amazon-linux-2-all-in-one
rules:
- when: manual
allow_failure: true
pr_extended:
extends: .kubevirt
stage: deploy-extended
rules:
- if: $PR_LABELS =~ /.*(ci-extended|ci-full).*/
when: on_success
- if: $CI_PIPELINE_SOURCE == "schedule" && $CI_PIPELINE_SCHEDULE_DESCRIPTION == "daily-ci"
when: on_success
- when: manual
allow_failure: true
parallel:
matrix:
- TESTCASE:
- almalinux9-calico
- almalinux9-calico-remove-node
- almalinux9-docker
- debian11-docker
- debian12-calico
- debian12-docker
- debian13-calico
- rockylinux9-calico
- rockylinux10-calico
- ubuntu22-all-in-one-docker
- ubuntu24-all-in-one-docker
- ubuntu24-calico-all-in-one
- ubuntu24-calico-etcd-kubeadm
- ubuntu24-flannel
# TODO: migrate to pr-full, fix the broken ones
periodic:
allow_failure: true
extends: .kubevirt
rules:
- if: $CI_PIPELINE_SOURCE == "schedule" && $CI_PIPELINE_SCHEDULE_DESCRIPTION == "daily-ci"
when: on_success
parallel:
matrix:
- TESTCASE:
- debian11-calico-upgrade
- debian11-calico-upgrade-once
- debian12-cilium-svc-proxy
- fedora39-calico-selinux
- fedora40-docker-calico
- fedora41-calico-selinux
- ubuntu24-calico-etcd-kubeadm-upgrade-ha
- ubuntu24-calico-ha-recover
- ubuntu24-calico-ha-recover-noquorum
================================================
FILE: .gitlab-ci/lint.yml
================================================
---
pre-commit:
stage: test
tags:
- ffci
image: 'ghcr.io/pre-commit-ci/runner-image@sha256:fe01a6ec51b298412990b88627c3973b1146c7304f930f469bafa29ba60bcde9'
variables:
PRE_COMMIT_HOME: ${CI_PROJECT_DIR}/.cache/pre-commit
ANSIBLE_STDOUT_CALLBACK: default
script:
- pre-commit run --all-files --show-diff-on-failure
cache:
key: pre-commit-2
paths:
- ${PRE_COMMIT_HOME}
when: 'always'
needs: []
vagrant-validate:
extends: .job
stage: test
tags: [ffci]
variables:
VAGRANT_VERSION: 2.3.7
script:
- ./tests/scripts/vagrant-validate.sh
================================================
FILE: .gitlab-ci/molecule.yml
================================================
---
.molecule:
tags: [ffci]
rules: # run on ci-short as well
- if: $CI_COMMIT_BRANCH =~ /^pr-.*$/
when: on_success
- if: $CI_PIPELINE_SOURCE == "schedule" && $CI_PIPELINE_SCHEDULE_DESCRIPTION == "daily-ci"
when: on_success
- when: manual
allow_failure: true
stage: deploy-part1
image: $PIPELINE_IMAGE
needs:
- pipeline-image
script:
- ./tests/scripts/molecule_run.sh
after_script:
- rm -fr molecule_logs
- mkdir -p molecule_logs
- find ~/.cache/molecule/ \( -name '*.out' -o -name '*.err' \) -type f | xargs tar -uf molecule_logs/molecule.tar
- gzip molecule_logs/molecule.tar
artifacts:
when: always
paths:
- molecule_logs/
molecule:
extends: .molecule
script:
- ./tests/scripts/molecule_run.sh -i $ROLE
parallel:
matrix:
- ROLE:
- container-engine/cri-dockerd
- container-engine/containerd
- container-engine/cri-o
- container-engine/gvisor
- container-engine/youki
- adduser
- bastion-ssh-config
- bootstrap_os
molecule_full:
allow_failure: true
rules:
- if: $CI_PIPELINE_SOURCE == "schedule" && $CI_PIPELINE_SCHEDULE_DESCRIPTION == "daily-ci"
when: on_success
- when: manual
allow_failure: true
extends: molecule
parallel:
matrix:
- ROLE:
# FIXME : tests below are perma-failing
- container-engine/kata-containers
================================================
FILE: .gitlab-ci/terraform.yml
================================================
---
# Tests for contrib/terraform/
.terraform_install:
extends: .job
needs:
- pipeline-image
variables:
TF_VAR_public_key_path: "${ANSIBLE_PRIVATE_KEY_FILE}.pub"
TF_VAR_ssh_private_key_path: $ANSIBLE_PRIVATE_KEY_FILE
CLUSTER: $CI_COMMIT_REF_NAME
TERRAFORM_STATE_ROOT: $CI_PROJECT_DIR
stage: deploy-part1
before_script:
- ./tests/scripts/rebase.sh
- mkdir -p cluster-dump $ANSIBLE_INVENTORY
- ./tests/scripts/opentofu_install.sh
- cp contrib/terraform/$PROVIDER/sample-inventory/cluster.tfvars .
- ln -rs -t $ANSIBLE_INVENTORY contrib/terraform/$PROVIDER/hosts
- tofu -chdir="contrib/terraform/$PROVIDER" init
terraform_validate:
extends: .terraform_install
tags: [ffci]
only: ['master', /^pr-.*$/]
script:
- tofu -chdir="contrib/terraform/$PROVIDER" validate
- tofu -chdir="contrib/terraform/$PROVIDER" fmt -check -diff
stage: test
needs:
- pipeline-image
parallel:
matrix:
- PROVIDER:
- openstack
- aws
- exoscale
- hetzner
- vsphere
- upcloud
.terraform_apply:
extends: .terraform_install
tags: [ffci]
stage: deploy-extended
when: manual
only: [/^pr-.*$/]
variables:
ANSIBLE_INVENTORY_UNPARSED_FAILED: "true"
ANSIBLE_REMOTE_USER: ubuntu # the openstack terraform module does not handle custom user correctly
ANSIBLE_SSH_RETRIES: 15
TF_VAR_ssh_user: $ANSIBLE_REMOTE_USER
TF_VAR_cluster_name: $CI_JOB_ID
script:
# Set Ansible config
- cp ansible.cfg ~/.ansible.cfg
- ssh-keygen -N '' -f $ANSIBLE_PRIVATE_KEY_FILE -t rsa
- mkdir -p contrib/terraform/$PROVIDER/group_vars
# Random subnet to avoid routing conflicts
- export TF_VAR_subnet_cidr="10.$(( $RANDOM % 256 )).$(( $RANDOM % 256 )).0/24"
- tofu -chdir="contrib/terraform/$PROVIDER" apply -auto-approve -parallelism=1
- tests/scripts/testcases_run.sh
after_script:
# Cleanup regardless of exit code
- tofu -chdir="contrib/terraform/$PROVIDER" destroy -auto-approve
# Elastx is generously donating resources for Kubespray on Openstack CI
# Contacts: @gix @bl0m1
.elastx_variables: &elastx_variables
OS_AUTH_URL: https://ops.elastx.cloud:5000
OS_PROJECT_ID: 564c6b461c6b44b1bb19cdb9c2d928e4
OS_PROJECT_NAME: kubespray_ci
OS_USER_DOMAIN_NAME: Default
OS_PROJECT_DOMAIN_ID: default
OS_USERNAME: kubespray@root314.com
OS_REGION_NAME: se-sto
OS_INTERFACE: public
OS_IDENTITY_API_VERSION: "3"
TF_VAR_router_id: "ab95917c-41fb-4881-b507-3a6dfe9403df"
tf-elastx_cleanup:
tags: [ffci]
image: python
variables:
<<: *elastx_variables
before_script:
- pip install -r scripts/openstack-cleanup/requirements.txt
script:
- ./scripts/openstack-cleanup/main.py
allow_failure: true
tf-elastx_ubuntu24-calico:
extends: .terraform_apply
stage: deploy-part1
when: on_success
variables:
<<: *elastx_variables
PROVIDER: openstack
ANSIBLE_TIMEOUT: "60"
TF_VAR_number_of_k8s_masters: "1"
TF_VAR_number_of_k8s_masters_no_floating_ip: "0"
TF_VAR_number_of_k8s_masters_no_floating_ip_no_etcd: "0"
TF_VAR_number_of_etcd: "0"
TF_VAR_number_of_k8s_nodes: "1"
TF_VAR_number_of_k8s_nodes_no_floating_ip: "0"
TF_VAR_number_of_gfs_nodes_no_floating_ip: "0"
TF_VAR_number_of_bastions: "0"
TF_VAR_number_of_k8s_masters_no_etcd: "0"
TF_VAR_floatingip_pool: "elx-public1"
TF_VAR_dns_nameservers: '["1.1.1.1", "8.8.8.8", "8.8.4.4"]'
TF_VAR_use_access_ip: "0"
TF_VAR_external_net: "600b8501-78cb-4155-9c9f-23dfcba88828"
TF_VAR_network_name: "ci-$CI_JOB_ID"
TF_VAR_az_list: '["sto1"]'
TF_VAR_az_list_node: '["sto1"]'
TF_VAR_flavor_k8s_master: 3f73fc93-ec61-4808-88df-2580d94c1a9b # v1-standard-2
TF_VAR_flavor_k8s_node: 3f73fc93-ec61-4808-88df-2580d94c1a9b # v1-standard-2
TF_VAR_image: ubuntu-24.04-server-latest
TF_VAR_k8s_allowed_remote_ips: '["0.0.0.0/0"]'
TESTCASE: $CI_JOB_NAME
================================================
FILE: .gitlab-ci/vagrant.yml
================================================
---
vagrant:
extends: .job-moderated
variables:
CI_PLATFORM: "vagrant"
SSH_USER: "vagrant"
VAGRANT_DEFAULT_PROVIDER: "libvirt"
KUBESPRAY_VAGRANT_CONFIG: tests/files/${TESTCASE}.rb
DOCKER_NAME: vagrant
VAGRANT_ANSIBLE_TAGS: facts
VAGRANT_HOME: "$CI_PROJECT_DIR/.vagrant.d"
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
tags: [ffci-vm-large]
image: quay.io/kubespray/vm-kubespray-ci:v13
services: []
before_script:
- echo $USER
- python3 -m venv citest
- source citest/bin/activate
- vagrant plugin expunge --reinstall --force --no-tty
- vagrant plugin install vagrant-libvirt
- pip install --no-compile --no-cache-dir pip -U
- pip install --no-compile --no-cache-dir -r $CI_PROJECT_DIR/requirements.txt
- pip install --no-compile --no-cache-dir -r $CI_PROJECT_DIR/tests/requirements.txt
- ./tests/scripts/vagrant_clean.sh
script:
- vagrant up
- ./tests/scripts/testcases_run.sh
after_script:
- vagrant destroy -f
cache:
key: $CI_JOB_NAME_SLUG
paths:
- .vagrant.d/boxes
- .cache/pip
policy: pull-push # TODO: change to "pull" when not on main
stage: deploy-extended
rules:
- if: $PR_LABELS =~ /.*ci-full.*/
when: on_success
- if: $CI_PIPELINE_SOURCE == "schedule" && $CI_PIPELINE_SCHEDULE_DESCRIPTION == "daily-ci"
when: on_success
- when: manual
allow_failure: true
parallel:
matrix:
- TESTCASE:
- ubuntu24-calico-dual-stack
- ubuntu24-calico-ipv6only-stack
================================================
FILE: .gitlab-ci.yml
================================================
---
stages:
- build # build docker image used in most other jobs
- test # unit tests
- deploy-part1 # kubespray runs - common setup
- deploy-extended # kubespray runs - rarer or costlier (to test) setups
variables:
FAILFASTCI_NAMESPACE: 'kargo-ci'
GITLAB_REPOSITORY: 'kargo-ci/kubernetes-sigs-kubespray'
GIT_CONFIG_COUNT: 2
GIT_CONFIG_KEY_0: user.email
GIT_CONFIG_VALUE_0: "ci@kubespray.io"
GIT_CONFIG_KEY_1: user.name
GIT_CONFIG_VALUE_1: "Kubespray CI"
ANSIBLE_FORCE_COLOR: "true"
MAGIC: "ci check this"
GS_ACCESS_KEY_ID: $GS_KEY
GS_SECRET_ACCESS_KEY: $GS_SECRET
CONTAINER_ENGINE: docker
GCE_PREEMPTIBLE: "false"
ANSIBLE_KEEP_REMOTE_FILES: "1"
ANSIBLE_CONFIG: ./tests/ansible.cfg
ANSIBLE_REMOTE_USER: kubespray
ANSIBLE_PRIVATE_KEY_FILE: /tmp/id_rsa
ANSIBLE_INVENTORY: /tmp/inventory
ANSIBLE_STDOUT_CALLBACK: "default"
RESET_CHECK: "false"
REMOVE_NODE_CHECK: "false"
UPGRADE_TEST: "false"
MITOGEN_ENABLE: "false"
ANSIBLE_VERBOSITY: 2
RECOVER_CONTROL_PLANE_TEST: "false"
RECOVER_CONTROL_PLANE_TEST_GROUPS: "etcd[2:]:kube_control_plane[1:]"
OPENTOFU_VERSION: v1.9.1
PIPELINE_IMAGE: "$CI_REGISTRY_IMAGE/pipeline:${CI_PIPELINE_ID}-${CI_COMMIT_SHORT_SHA}"
before_script:
- ./tests/scripts/rebase.sh
- mkdir -p cluster-dump $ANSIBLE_INVENTORY
.job: &job
tags:
- ffci
image: $PIPELINE_IMAGE
artifacts:
when: always
paths:
- cluster-dump/
needs:
- pipeline-image
.job-moderated:
extends: .job
needs:
- pipeline-image
- pre-commit # lint
- vagrant-validate # lint
include:
- .gitlab-ci/build.yml
- .gitlab-ci/lint.yml
- .gitlab-ci/terraform.yml
- .gitlab-ci/kubevirt.yml
- .gitlab-ci/vagrant.yml
- .gitlab-ci/molecule.yml
================================================
FILE: .gitmodules
================================================
================================================
FILE: .md_style.rb
================================================
all
exclude_rule 'MD013'
exclude_rule 'MD029'
rule 'MD007', :indent => 2
================================================
FILE: .mdlrc
================================================
style "#{File.dirname(__FILE__)}/.md_style.rb"
================================================
FILE: .nojekyll
================================================
================================================
FILE: .pre-commit-config.yaml
================================================
---
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: check-added-large-files
- id: check-case-conflict
- id: check-executables-have-shebangs
- id: check-xml
- id: check-merge-conflict
- id: detect-private-key
- id: end-of-file-fixer
- id: forbid-new-submodules
- id: requirements-txt-fixer
- id: trailing-whitespace
- repo: https://github.com/adrienverge/yamllint.git
rev: v1.37.1
hooks:
- id: yamllint
args: [--strict]
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: v0.11.0.1
hooks:
- id: shellcheck
args: ["--severity=error"]
exclude: "^.git"
files: "\\.sh$"
- repo: https://github.com/ansible/ansible-lint
rev: v25.11.0
hooks:
- id: ansible-lint
additional_dependencies:
- jmespath==1.0.1
- netaddr==1.3.0
- distlib
- repo: https://github.com/golangci/misspell
rev: v0.7.0
hooks:
- id: misspell
exclude: "OWNERS_ALIASES$"
- repo: local
hooks:
- id: collection-build-install
name: Build and install kubernetes-sigs.kubespray Ansible collection
language: python
additional_dependencies:
- ansible-core>=2.16.4
- distlib
entry: tests/scripts/collection-build-install.sh
pass_filenames: false
- id: generate-docs-sidebar
name: generate-docs-sidebar
entry: scripts/gen_docs_sidebar.sh
language: script
pass_filenames: false
- id: ci-matrix
name: ci-matrix
entry: tests/scripts/md-table/main.py
language: python
pass_filenames: false
additional_dependencies:
- jinja2
- pathlib
- pyaml
- id: check-galaxy-version
name: Verify correct version for galaxy.yml
entry: scripts/galaxy_version.py
language: python
pass_filenames: false
additional_dependencies:
- ruamel.yaml
- id: jinja-syntax-check
name: jinja-syntax-check
entry: tests/scripts/check-templates.py
language: python
types:
- jinja
additional_dependencies:
- jinja2
- id: propagate-ansible-variables
name: Update static files referencing default kubespray values
language: python
additional_dependencies:
- ansible-core>=2.16.4
entry: scripts/propagate_ansible_variables.yml
pass_filenames: false
- id: check-checksums-sorted
name: Check that our checksums are correctly sorted by version
entry: scripts/assert-sorted-checksums.yml
language: python
pass_filenames: false
additional_dependencies:
- ansible
- repo: https://github.com/markdownlint/markdownlint
rev: v0.12.0
hooks:
- id: markdownlint
exclude: "^.github|(^docs/_sidebar\\.md$)"
================================================
FILE: .yamllint
================================================
---
extends: default
ignore: |
.git/
.github/
# Generated file
tests/files/custom_cni/cilium.yaml
# https://ansible.readthedocs.io/projects/lint/rules/yaml/
rules:
braces:
min-spaces-inside: 0
max-spaces-inside: 1
brackets:
min-spaces-inside: 0
max-spaces-inside: 1
comments:
min-spaces-from-content: 1
# https://github.com/adrienverge/yamllint/issues/384
comments-indentation: false
indentation:
spaces: 2
indent-sequences: consistent
line-length: disable
new-line-at-end-of-file: disable
octal-values:
forbid-implicit-octal: true # yamllint defaults to false
forbid-explicit-octal: true # yamllint defaults to false
================================================
FILE: CHANGELOG.md
================================================
# See our release notes on [GitHub](https://github.com/kubernetes-sigs/kubespray/releases)
================================================
FILE: CNAME
================================================
kubespray.io
================================================
FILE: CONTRIBUTING.md
================================================
# Contributing guidelines
## How to become a contributor and submit your own code
### Environment setup
It is recommended to use filter to manage the GitHub email notification, see [examples for setting filters to Kubernetes Github notifications](https://github.com/kubernetes/community/blob/master/communication/best-practices.md#examples-for-setting-filters-to-kubernetes-github-notifications)
To install development dependencies you can set up a python virtual env with the necessary dependencies:
```ShellSession
virtualenv venv
source venv/bin/activate
pip install -r tests/requirements.txt
```
#### Linting
Kubespray uses [pre-commit](https://pre-commit.com) hook configuration to run several linters, please install this tool and use it to run validation tests before submitting a PR.
```ShellSession
pre-commit install
pre-commit run -a # To run pre-commit hook on all files in the repository, even if they were not modified
```
#### Molecule
[molecule](https://github.com/ansible-community/molecule) is designed to help the development and testing of Ansible roles. In Kubespray you can run it all for all roles with `./tests/scripts/molecule_run.sh` or for a specific role (that you are working with) with `molecule test` from the role directory (`cd roles/my-role`).
When developing or debugging a role it can be useful to run `molecule create` and `molecule converge` separately. Then you can use `molecule login` to SSH into the test environment.
#### Vagrant
Vagrant with VirtualBox or libvirt driver helps you to quickly spin test clusters to test things end to end. See [README.md#vagrant](README.md)
### Contributing A Patch
1. Submit an issue describing your proposed change to the repo in question.
2. The [repo owners](OWNERS) will respond to your issue promptly.
3. Fork the desired repo, develop and test your code changes.
4. Install [pre-commit](https://pre-commit.com) and install it in your development repo.
5. Address any pre-commit validation failures.
6. Sign the CNCF CLA (<https://git.k8s.io/community/CLA.md#the-contributor-license-agreement>)
7. Submit a pull request.
8. Work with the reviewers on their suggestions.
9. Ensure to rebase to the HEAD of your target branch and squash un-necessary commits (<https://blog.carbonfive.com/always-squash-and-rebase-your-git-commits/>) before final merger of your contribution.
================================================
FILE: Dockerfile
================================================
# syntax=docker/dockerfile:1
# Use immutable image tags rather than mutable tags (like ubuntu:24.04)
FROM ubuntu:noble-20260113@sha256:cd1dba651b3080c3686ecf4e3c4220f026b521fb76978881737d24f200828b2b
# Some tools like yamllint need this
# Pip needs this as well at the moment to install ansible
# (and potentially other packages)
# See: https://github.com/pypa/pip/issues/10219
ENV LANG=C.UTF-8 \
DEBIAN_FRONTEND=noninteractive \
PYTHONDONTWRITEBYTECODE=1
WORKDIR /kubespray
# hadolint ignore=DL3008
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
apt-get update -q \
&& apt-get install -yq --no-install-recommends \
curl \
python3 \
python3-pip \
sshpass \
vim \
rsync \
openssh-client \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /var/log/*
RUN --mount=type=bind,source=requirements.txt,target=requirements.txt \
--mount=type=cache,sharing=locked,id=pipcache,mode=0777,target=/root/.cache/pip \
pip install --break-system-packages --no-compile --no-cache-dir -r requirements.txt \
&& find /usr -type d -name '*__pycache__' -prune -exec rm -rf {} \;
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN OS_ARCHITECTURE=$(dpkg --print-architecture) \
&& curl -L "https://dl.k8s.io/release/v1.35.1/bin/linux/${OS_ARCHITECTURE}/kubectl" -o /usr/local/bin/kubectl \
&& echo "$(curl -L "https://dl.k8s.io/release/v1.35.1/bin/linux/${OS_ARCHITECTURE}/kubectl.sha256")" /usr/local/bin/kubectl | sha256sum --check \
&& chmod a+x /usr/local/bin/kubectl
COPY *.yml ./
COPY *.cfg ./
COPY roles ./roles
COPY contrib ./contrib
COPY inventory ./inventory
COPY library ./library
COPY extra_playbooks ./extra_playbooks
COPY playbooks ./playbooks
COPY plugins ./plugins
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2016 Kubespray
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: OWNERS
================================================
# See the OWNERS docs at https://go.k8s.io/owners
approvers:
- kubespray-approvers
reviewers:
- kubespray-reviewers
emeritus_approvers:
- kubespray-emeritus_approvers
================================================
FILE: OWNERS_ALIASES
================================================
aliases:
kubespray-approvers:
- ant31
- mzaian
- tico88612
- vannten
- yankay
kubespray-reviewers:
- cyclinder
- erikjiang
- mzaian
- tico88612
- vannten
- yankay
kubespray-emeritus_approvers:
- atoms
- chadswen
- cristicalin
- floryut
- liupeng0518
- luckysb
- mattymo
- miouge1
- oomichi
- riverzhang
- woopstar
================================================
FILE: README.md
================================================
# Deploy a Production Ready Kubernetes Cluster
If you have questions, check the documentation at [kubespray.io](https://kubespray.io) and join us on the [kubernetes slack](https://kubernetes.slack.com), channel **\#kubespray**.
You can get your invite [here](http://slack.k8s.io/)
- Can be deployed on **[AWS](docs/cloud_providers/aws.md), GCE, [Azure](docs/cloud_providers/azure.md), [OpenStack](docs/cloud_controllers/openstack.md), [vSphere](docs/cloud_controllers/vsphere.md), [Equinix Metal](docs/cloud_providers/equinix-metal.md) (bare metal), Oracle Cloud Infrastructure (Experimental), or Baremetal**
- **Highly available** cluster
- **Composable** (Choice of the network plugin for instance)
- Supports most popular **Linux distributions**
- **Continuous integration tests**
## Quick Start
Below are several ways to use Kubespray to deploy a Kubernetes cluster.
### Docker
Ensure you have installed Docker then
```ShellSession
docker run --rm -it --mount type=bind,source="$(pwd)"/inventory/sample,dst=/inventory \
--mount type=bind,source="${HOME}"/.ssh/id_rsa,dst=/root/.ssh/id_rsa \
quay.io/kubespray/kubespray:v2.30.0 bash
# Inside the container you may now run the kubespray playbooks:
ansible-playbook -i /inventory/inventory.ini --private-key /root/.ssh/id_rsa cluster.yml
```
### Ansible
#### Usage
See [Getting started](/docs/getting_started/getting-started.md)
#### Collection
See [here](docs/ansible/ansible_collection.md) if you wish to use this repository as an Ansible collection
### Vagrant
For Vagrant we need to install Python dependencies for provisioning tasks.
Check that ``Python`` and ``pip`` are installed:
```ShellSession
python -V && pip -V
```
If this returns the version of the software, you're good to go. If not, download and install Python from here <https://www.python.org/downloads/source/>
Install Ansible according to [Ansible installation guide](/docs/ansible/ansible.md#installing-ansible)
then run the following step:
```ShellSession
vagrant up
```
## Documents
- [Requirements](#requirements)
- [Kubespray vs ...](docs/getting_started/comparisons.md)
- [Getting started](docs/getting_started/getting-started.md)
- [Setting up your first cluster](docs/getting_started/setting-up-your-first-cluster.md)
- [Ansible inventory and tags](docs/ansible/ansible.md)
- [Integration with existing ansible repo](docs/operations/integration.md)
- [Deployment data variables](docs/ansible/vars.md)
- [DNS stack](docs/advanced/dns-stack.md)
- [HA mode](docs/operations/ha-mode.md)
- [Network plugins](#network-plugins)
- [Vagrant install](docs/developers/vagrant.md)
- [Flatcar Container Linux bootstrap](docs/operating_systems/flatcar.md)
- [Fedora CoreOS bootstrap](docs/operating_systems/fcos.md)
- [openSUSE setup](docs/operating_systems/opensuse.md)
- [Downloaded artifacts](docs/advanced/downloads.md)
- [Equinix Metal](docs/cloud_providers/equinix-metal.md)
- [OpenStack](docs/cloud_controllers/openstack.md)
- [vSphere](docs/cloud_controllers/vsphere.md)
- [Large deployments](docs/operations/large-deployments.md)
- [Adding/replacing a node](docs/operations/nodes.md)
- [Upgrades basics](docs/operations/upgrades.md)
- [Air-Gap installation](docs/operations/offline-environment.md)
- [NTP](docs/advanced/ntp.md)
- [Hardening](docs/operations/hardening.md)
- [Mirror](docs/operations/mirror.md)
- [Roadmap](docs/roadmap/roadmap.md)
## Supported Linux Distributions
- **Flatcar Container Linux by Kinvolk**
- **Debian** Bookworm, Bullseye, Trixie
- **Ubuntu** 22.04, 24.04
- **CentOS Stream / RHEL** 9, 10
- **Fedora** 39, 40, 41, 42
- **Fedora CoreOS** (see [fcos Note](docs/operating_systems/fcos.md))
- **openSUSE** Leap 15.x/Tumbleweed
- **Oracle Linux** 9, 10
- **Alma Linux** 9, 10
- **Rocky Linux** 9, 10 (experimental in 10: see [Rocky Linux 10 notes](docs/operating_systems/rhel.md#rocky-linux-10))
- **Kylin Linux Advanced Server V10** (experimental: see [kylin linux notes](docs/operating_systems/kylinlinux.md))
- **Amazon Linux 2** (experimental: see [amazon linux notes](docs/operating_systems/amazonlinux.md))
- **UOS Linux** (experimental: see [uos linux notes](docs/operating_systems/uoslinux.md))
- **openEuler** (experimental: see [openEuler notes](docs/operating_systems/openeuler.md))
Note:
- Upstart/SysV init based OS types are not supported.
- [Kernel requirements](docs/operations/kernel-requirements.md) (please read if the OS kernel version is < 4.19).
## Supported Components
<!-- BEGIN ANSIBLE MANAGED BLOCK -->
- Core
- [kubernetes](https://github.com/kubernetes/kubernetes) 1.35.1
- [etcd](https://github.com/etcd-io/etcd) 3.6.8
- [docker](https://www.docker.com/) 28.3
- [containerd](https://containerd.io/) 2.2.1
- [cri-o](http://cri-o.io/) 1.35.0 (experimental: see [CRI-O Note](docs/CRI/cri-o.md). Only on fedora, ubuntu and centos based OS)
- Network Plugin
- [cni-plugins](https://github.com/containernetworking/plugins) 1.8.0
- [calico](https://github.com/projectcalico/calico) 3.30.6
- [cilium](https://github.com/cilium/cilium) 1.19.1
- [flannel](https://github.com/flannel-io/flannel) 0.27.3
- [kube-ovn](https://github.com/alauda/kube-ovn) 1.12.21
- [kube-router](https://github.com/cloudnativelabs/kube-router) 2.1.1
- [multus](https://github.com/k8snetworkplumbingwg/multus-cni) 4.2.2
- [kube-vip](https://github.com/kube-vip/kube-vip) 1.0.3
- Application
- [cert-manager](https://github.com/jetstack/cert-manager) 1.15.3
- [coredns](https://github.com/coredns/coredns) 1.12.4
- [argocd](https://argoproj.github.io/) 2.14.5
- [helm](https://helm.sh/) 3.18.4
- [metallb](https://metallb.universe.tf/) 0.13.9
- [registry](https://github.com/distribution/distribution) 2.8.1
- Storage Plugin
- [aws-ebs-csi-plugin](https://github.com/kubernetes-sigs/aws-ebs-csi-driver) 0.5.0
- [azure-csi-plugin](https://github.com/kubernetes-sigs/azuredisk-csi-driver) 1.10.0
- [cinder-csi-plugin](https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/cinder-csi-plugin/using-cinder-csi-plugin.md) 1.30.0
- [gcp-pd-csi-plugin](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver) 1.9.2
- [local-path-provisioner](https://github.com/rancher/local-path-provisioner) 0.0.32
- [local-volume-provisioner](https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner) 2.5.0
- [node-feature-discovery](https://github.com/kubernetes-sigs/node-feature-discovery) 0.16.4
<!-- END ANSIBLE MANAGED BLOCK -->
## Container Runtime Notes
- The cri-o version should be aligned with the respective kubernetes version (i.e. kube_version=1.20.x, crio_version=1.20)
## Requirements
- **Minimum required version of Kubernetes is v1.30**
- **Ansible v2.14+, Jinja 2.11+ and python-netaddr is installed on the machine that will run Ansible commands**
- The target servers must have **access to the Internet** in order to pull docker images. Otherwise, additional configuration is required (See [Offline Environment](docs/operations/offline-environment.md))
- The target servers are configured to allow **IPv4 forwarding**.
- If using IPv6 for pods and services, the target servers are configured to allow **IPv6 forwarding**.
- The **firewalls are not managed**, you'll need to implement your own rules the way you used to.
in order to avoid any issue during deployment you should disable your firewall.
- If kubespray is run from non-root user account, correct privilege escalation method
should be configured in the target servers. Then the `ansible_become` flag
or command parameters `--become or -b` should be specified.
Hardware:
These limits are safeguarded by Kubespray. Actual requirements for your workload can differ. For a sizing guide go to the [Building Large Clusters](https://kubernetes.io/docs/setup/cluster-large/#size-of-master-and-master-components) guide.
- Control Plane
- Memory: 2 GB
- Worker Node
- Memory: 1 GB
## Network Plugins
You can choose among ten network plugins. (default: `calico`, except Vagrant uses `flannel`)
- [flannel](docs/CNI/flannel.md): gre/vxlan (layer 2) networking.
- [Calico](https://docs.tigera.io/calico/latest/about/) is a networking and network policy provider. Calico supports a flexible set of networking options
designed to give you the most efficient networking across a range of situations, including non-overlay
and overlay networks, with or without BGP. Calico uses the same engine to enforce network policy for hosts,
pods, and (if using Istio and Envoy) applications at the service mesh layer.
- [cilium](http://docs.cilium.io/en/latest/): layer 3/4 networking (as well as layer 7 to protect and secure application protocols), supports dynamic insertion of BPF bytecode into the Linux kernel to implement security services, networking and visibility logic.
- [kube-ovn](docs/CNI/kube-ovn.md): Kube-OVN integrates the OVN-based Network Virtualization with Kubernetes. It offers an advanced Container Network Fabric for Enterprises.
- [kube-router](docs/CNI/kube-router.md): Kube-router is a L3 CNI for Kubernetes networking aiming to provide operational
simplicity and high performance: it uses IPVS to provide Kube Services Proxy (if setup to replace kube-proxy),
iptables for network policies, and BGP for ods L3 networking (with optionally BGP peering with out-of-cluster BGP peers).
It can also optionally advertise routes to Kubernetes cluster Pods CIDRs, ClusterIPs, ExternalIPs and LoadBalancerIPs.
- [macvlan](docs/CNI/macvlan.md): Macvlan is a Linux network driver. Pods have their own unique Mac and Ip address, connected directly the physical (layer 2) network.
- [multus](docs/CNI/multus.md): Multus is a meta CNI plugin that provides multiple network interface support to pods. For each interface Multus delegates CNI calls to secondary CNI plugins such as Calico, macvlan, etc.
- [custom_cni](roles/network-plugin/custom_cni/) : You can specify some manifests that will be applied to the clusters to bring you own CNI and use non-supported ones by Kubespray.
See `tests/files/custom_cni/README.md` and `tests/files/custom_cni/values.yaml`for an example with a CNI provided by a Helm Chart.
The network plugin to use is defined by the variable `kube_network_plugin`. There is also an
option to leverage built-in cloud provider networking instead.
See also [Network checker](docs/advanced/netcheck.md).
## Ingress Plugins
- [metallb](docs/ingress/metallb.md): the MetalLB bare-metal service LoadBalancer provider.
## Community docs and resources
- [kubernetes.io/docs/setup/production-environment/tools/kubespray/](https://kubernetes.io/docs/setup/production-environment/tools/kubespray/)
- [kubespray, monitoring and logging](https://github.com/gregbkr/kubernetes-kargo-logging-monitoring) by @gregbkr
- [Deploy Kubernetes w/ Ansible & Terraform](https://rsmitty.github.io/Terraform-Ansible-Kubernetes/) by @rsmitty
- [Deploy a Kubernetes Cluster with Kubespray (video)](https://www.youtube.com/watch?v=CJ5G4GpqDy0)
## Tools and projects on top of Kubespray
- [Digital Rebar Provision](https://github.com/digitalrebar/provision/blob/v4/doc/integrations/ansible.rst)
- [Terraform Contrib](https://github.com/kubernetes-sigs/kubespray/tree/master/contrib/terraform)
- [Kubean](https://github.com/kubean-io/kubean)
## CI Tests
[](https://gitlab.com/kargo-ci/kubernetes-sigs-kubespray/-/pipelines)
CI/end-to-end tests sponsored by: [CNCF](https://cncf.io), [Equinix Metal](https://metal.equinix.com/), [OVHcloud](https://www.ovhcloud.com/), [ELASTX](https://elastx.se/).
See the [test matrix](docs/developers/test_cases.md) for details.
================================================
FILE: RELEASE.md
================================================
# Release Process
The Kubespray Project is released on an as-needed basis. The process is as follows:
1. An issue is proposing a new release with a changelog since the last release. Please see [a good sample issue](https://github.com/kubernetes-sigs/kubespray/issues/8325)
1. At least one of the [approvers](OWNERS_ALIASES) must approve this release
1. (Only for major releases) The `kube_version_min_required` variable is set to `n-1`
1. (Only for major releases) Remove hashes for [EOL versions](https://github.com/kubernetes/website/blob/main/content/en/releases/patch-releases.md) of kubernetes from `*_checksums` variables.
1. Create the release note with [Kubernetes Release Notes Generator](https://github.com/kubernetes/release/blob/master/cmd/release-notes/README.md). See the following `Release note creation` section for the details.
1. An approver creates [new release in GitHub](https://github.com/kubernetes-sigs/kubespray/releases/new) using a version and tag name like `vX.Y.Z` and attaching the release notes
1. (Only for major releases) An approver creates a release branch in the form `release-X.Y`
1. (For major releases) On the `master` branch: bump the version in `galaxy.yml` to the next expected major release (X.y.0 with y = Y + 1), make a Pull Request.
1. (For minor releases) On the `release-X.Y` branch: bump the version in `galaxy.yml` to the next expected minor release (X.Y.z with z = Z + 1), make a Pull Request.
1. The corresponding version of [quay.io/kubespray/kubespray:vX.Y.Z](https://quay.io/repository/kubespray/kubespray) and [quay.io/kubespray/vagrant:vX.Y.Z](https://quay.io/repository/kubespray/vagrant) container images are built and tagged. See the following `Container image creation` section for the details.
1. The release issue is closed
1. An announcement email is sent to `dev@kubernetes.io` with the subject `[ANNOUNCE] Kubespray $VERSION is released`
1. The topic of the #kubespray channel is updated with `vX.Y.Z is released! | ...`
1. Create/Update Issue for upgrading kubernetes and [k8s-conformance](https://github.com/cncf/k8s-conformance)
## Major/minor releases and milestones
* For major releases (vX.Y) Kubespray maintains one branch (`release-X.Y`). Minor releases (vX.Y.Z) are available only as tags.
* Security patches and bugs might be backported.
* Fixes for major releases (vX.Y) and minor releases (vX.Y.Z) are delivered
via maintenance releases (vX.Y.Z) and assigned to the corresponding open
[GitHub milestone](https://github.com/kubernetes-sigs/kubespray/milestones).
That milestone remains open for the major/minor releases support lifetime,
which ends once the milestone is closed. Then only a next major or minor release
can be done.
* Kubespray major and minor releases are bound to the given `kube_version` major/minor
version numbers and other components' arbitrary versions, like etcd or network plugins.
Older or newer component versions are not supported and not tested for the given
release (even if included in the checksum variables, like `kubeadm_checksums`).
* There is no unstable releases and no APIs, thus Kubespray doesn't follow
[semver](https://semver.org/). Every version describes only a stable release.
Breaking changes, if any introduced by changed defaults or non-contrib ansible roles'
playbooks, shall be described in the release notes. Other breaking changes, if any in
the contributed addons or bound versions of Kubernetes and other components, are
considered out of Kubespray scope and are up to the components' teams to deal with and
document.
* Minor releases can change components' versions, but not the major `kube_version`.
Greater `kube_version` requires a new major or minor release. For example, if Kubespray v2.0.0
is bound to `kube_version: 1.4.x`, `calico_version: 0.22.0`, `etcd_version: 3.0.6`,
then Kubespray v2.1.0 may be bound to only minor changes to `kube_version`, like v1.5.1
and *any* changes to other components, like etcd v4, or calico 1.2.3.
And Kubespray v3.x.x shall be bound to `kube_version: 2.x.x` respectively.
## Release note creation
You can create a release note with:
```shell
export GITHUB_TOKEN=<your-github-token>
export ORG=kubernetes-sigs
export REPO=kubespray
release-notes --start-sha <The start commit-id> --end-sha <The end commit-id> --dependencies=false --output=/tmp/kubespray-release-note --required-author=""
```
If the release note file(/tmp/kubespray-release-note) contains "### Uncategorized" pull requests, those pull requests don't have a valid kind label(`kind/feature`, etc.).
It is necessary to put a valid label on each pull request and run the above release-notes command again to get a better release note
## Container image creation
The container image `quay.io/kubespray/kubespray:vX.Y.Z` can be created from Dockerfile of the kubespray root directory:
```shell
cd kubespray/
nerdctl build -t quay.io/kubespray/kubespray:vX.Y.Z .
nerdctl push quay.io/kubespray/kubespray:vX.Y.Z
```
The container image `quay.io/kubespray/vagrant:vX.Y.Z` can be created from build.sh of test-infra/vagrant-docker/:
```shell
cd kubespray/test-infra/vagrant-docker/
./build vX.Y.Z
```
Please note that the above operation requires the permission to push container images into quay.io/kubespray/.
If you don't have the permission, please ask it on the #kubespray-dev channel.
================================================
FILE: SECURITY_CONTACTS
================================================
# Defined below are the security contacts for this repo.
#
# They are the contact point for the Product Security Committee to reach out
# to for triaging and handling of incoming issues.
#
# The below names agree to abide by the
# [Embargo Policy](https://git.k8s.io/security/private-distributors-list.md#embargo-policy)
# and will be removed and replaced if they violate that agreement.
#
# DO NOT REPORT SECURITY VULNERABILITIES DIRECTLY TO THESE NAMES, FOLLOW THE
# INSTRUCTIONS AT https://kubernetes.io/security/
floryut
ant31
VannTen
yankay
================================================
FILE: Vagrantfile
================================================
# -*- mode: ruby -*-
# # vi: set ft=ruby :
# For help on using kubespray with vagrant, check out docs/developers/vagrant.md
require 'fileutils'
require 'ipaddr'
require 'socket'
Vagrant.require_version ">= 2.0.0"
CONFIG = File.join(File.dirname(__FILE__), ENV['KUBESPRAY_VAGRANT_CONFIG'] || 'vagrant/config.rb')
FLATCAR_URL_TEMPLATE = "https://%s.release.flatcar-linux.net/amd64-usr/current/flatcar_production_vagrant.json"
# Uniq disk UUID for libvirt
DISK_UUID = Time.now.utc.to_i
SUPPORTED_OS = {
"flatcar-stable" => {box: "flatcar-stable", user: "core", box_url: FLATCAR_URL_TEMPLATE % ["stable"]},
"flatcar-beta" => {box: "flatcar-beta", user: "core", box_url: FLATCAR_URL_TEMPLATE % ["beta"]},
"flatcar-alpha" => {box: "flatcar-alpha", user: "core", box_url: FLATCAR_URL_TEMPLATE % ["alpha"]},
"flatcar-edge" => {box: "flatcar-edge", user: "core", box_url: FLATCAR_URL_TEMPLATE % ["edge"]},
"ubuntu2004" => {box: "generic/ubuntu2004", user: "vagrant"},
"ubuntu2204" => {box: "generic/ubuntu2204", user: "vagrant"},
"ubuntu2404" => {box: "bento/ubuntu-24.04", user: "vagrant"},
"centos8" => {box: "centos/8", user: "vagrant"},
"centos8-bento" => {box: "bento/centos-8", user: "vagrant"},
"almalinux8" => {box: "almalinux/8", user: "vagrant"},
"almalinux8-bento" => {box: "bento/almalinux-8", user: "vagrant"},
"almalinux9" => {box: "almalinux/9", user: "vagrant"},
"rockylinux8" => {box: "rockylinux/8", user: "vagrant"},
"rockylinux9" => {box: "rockylinux/9", user: "vagrant"},
"fedora39" => {box: "fedora/39-cloud-base", user: "vagrant"},
"fedora40" => {box: "fedora/40-cloud-base", user: "vagrant"},
"fedora39-arm64" => {box: "bento/fedora-39-arm64", user: "vagrant"},
"fedora40-arm64" => {box: "bento/fedora-40", user: "vagrant"},
"fedora41" => {box: "fedora/41-cloud-base", user: "vagrant"},
"fedora42" => {box: "fedora/42-cloud-base", user: "vagrant"},
"fedora41-bento" => {box: "bento/fedora-41", user: "vagrant"},
"opensuse" => {box: "opensuse/Leap-15.6.x86_64", user: "vagrant"},
"opensuse-tumbleweed" => {box: "opensuse/Tumbleweed.x86_64", user: "vagrant"},
"oraclelinux" => {box: "generic/oracle7", user: "vagrant"},
"oraclelinux8" => {box: "generic/oracle8", user: "vagrant"},
"rhel8" => {box: "generic/rhel8", user: "vagrant"},
"debian11" => {box: "debian/bullseye64", user: "vagrant"},
"debian12" => {box: "debian/bookworm64", user: "vagrant"},
}
if File.exist?(CONFIG)
require CONFIG
end
# Defaults for config options defined in CONFIG
$num_instances ||= 3
$instance_name_prefix ||= "k8s"
$vm_gui ||= false
$vm_memory ||= 2048
$vm_cpus ||= 2
$shared_folders ||= {}
$forwarded_ports ||= {}
$subnet ||= "172.18.8"
$subnet_ipv6 ||= "fd3c:b398:0698:0756"
$os ||= "ubuntu2004"
$network_plugin ||= "flannel"
$inventories ||= []
# Setting multi_networking to true will install Multus: https://github.com/k8snetworkplumbingwg/multus-cni
$multi_networking ||= "False"
$download_run_once ||= "True"
$download_force_cache ||= "False"
# Modify those to have separate groups (for instance, to test separate etcd:)
# first_control_plane = 1
# first_etcd = 4
# control_plane_instances = 3
# etcd_instances = 3
$first_node ||= 1
$first_control_plane ||= 1
$first_etcd ||= 1
# The first three nodes are etcd servers
$etcd_instances ||= [$num_instances, 3].min
# The first two nodes are kube masters
$control_plane_instances ||= [$num_instances, 2].min
# All nodes are kube nodes
$kube_node_instances ||= $num_instances - $first_node + 1
# The following only works when using the libvirt provider
$kube_node_instances_with_disks ||= false
$kube_node_instances_with_disks_size ||= "20G"
$kube_node_instances_with_disks_number ||= 2
$override_disk_size ||= false
$disk_size ||= "20GB"
$local_path_provisioner_enabled ||= "False"
$local_path_provisioner_claim_root ||= "/opt/local-path-provisioner/"
$libvirt_nested ||= false
# boolean or string (e.g. "-vvv")
$ansible_verbosity ||= false
$ansible_tags ||= ENV['VAGRANT_ANSIBLE_TAGS'] || ""
$vagrant_dir ||= File.join(File.dirname(__FILE__), ".vagrant")
$playbook ||= "cluster.yml"
$extra_vars ||= {}
host_vars = {}
def collect_networks(subnet, subnet_ipv6)
Socket.getifaddrs.filter_map do |iface|
next unless iface&.netmask&.ip_address && iface.addr
is_ipv6 = iface.addr.ipv6?
ip = IPAddr.new(iface.addr.ip_address.split('%').first)
ip_test = is_ipv6 ? IPAddr.new("#{subnet_ipv6}::0") : IPAddr.new("#{subnet}.0")
prefix = IPAddr.new(iface.netmask.ip_address).to_i.to_s(2).count('1')
network = ip.mask(prefix)
[IPAddr.new("#{network}/#{prefix}"), ip_test]
end
end
def subnet_in_use?(network_ips)
network_ips.any? { |net, test_ip| net.include?(test_ip) && test_ip != net }
end
network_ips = collect_networks($subnet, $subnet_ipv6)
if subnet_in_use?(network_ips)
puts "Invalid subnet provided, subnet is already in use: #{$subnet}.0"
puts "Subnets in use: #{network_ips.inspect}"
exit 1
end
# throw error if os is not supported
if ! SUPPORTED_OS.key?($os)
puts "Unsupported OS: #{$os}"
puts "Supported OS are: #{SUPPORTED_OS.keys.join(', ')}"
exit 1
end
$box = SUPPORTED_OS[$os][:box]
if Vagrant.has_plugin?("vagrant-proxyconf")
$no_proxy = ENV['NO_PROXY'] || ENV['no_proxy'] || "127.0.0.1,localhost"
(1..$num_instances).each do |i|
$no_proxy += ",#{$subnet}.#{i+100}"
end
end
Vagrant.configure("2") do |config|
config.vm.box = $box
if SUPPORTED_OS[$os].has_key? :box_url
config.vm.box_url = SUPPORTED_OS[$os][:box_url]
end
config.ssh.username = SUPPORTED_OS[$os][:user]
# plugin conflict
if Vagrant.has_plugin?("vagrant-vbguest") then
config.vbguest.auto_update = false
end
# always use Vagrants insecure key
config.ssh.insert_key = false
if ($override_disk_size)
unless Vagrant.has_plugin?("vagrant-disksize")
system "vagrant plugin install vagrant-disksize"
end
config.disksize.size = $disk_size
end
(1..$num_instances).each do |i|
config.vm.define vm_name = "%s-%01d" % [$instance_name_prefix, i] do |node|
node.vm.hostname = vm_name
if Vagrant.has_plugin?("vagrant-proxyconf")
node.proxy.http = ENV['HTTP_PROXY'] || ENV['http_proxy'] || ""
node.proxy.https = ENV['HTTPS_PROXY'] || ENV['https_proxy'] || ""
node.proxy.no_proxy = $no_proxy
end
["vmware_fusion", "vmware_workstation"].each do |vmware|
node.vm.provider vmware do |v|
v.vmx['memsize'] = $vm_memory
v.vmx['numvcpus'] = $vm_cpus
end
end
node.vm.provider :virtualbox do |vb|
vb.memory = $vm_memory
vb.cpus = $vm_cpus
vb.gui = $vm_gui
vb.linked_clone = true
vb.customize ["modifyvm", :id, "--vram", "8"] # ubuntu defaults to 256 MB which is a waste of precious RAM
vb.customize ["modifyvm", :id, "--audio", "none"]
end
node.vm.provider :libvirt do |lv|
lv.nested = $libvirt_nested
lv.cpu_mode = "host-model"
lv.memory = $vm_memory
lv.cpus = $vm_cpus
lv.default_prefix = 'kubespray'
# Fix kernel panic on fedora 28
if $os == "fedora"
lv.cpu_mode = "host-passthrough"
end
end
if $kube_node_instances_with_disks
# Libvirt
driverletters = ('a'..'z').to_a
node.vm.provider :libvirt do |lv|
# always make /dev/sd{a/b/c} so that CI can ensure that
# virtualbox and libvirt will have the same devices to use for OSDs
(1..$kube_node_instances_with_disks_number).each do |d|
lv.storage :file, :device => "hd#{driverletters[d]}", :path => "disk-#{i}-#{d}-#{DISK_UUID}.disk", :size => $kube_node_instances_with_disks_size, :bus => "scsi"
end
end
node.vm.provider :virtualbox do |vb|
# always make /dev/sd{a/b/c} so that CI can ensure that
# virtualbox and libvirt will have the same devices to use for OSDs
(1..$kube_node_instances_with_disks_number).each do |d|
vb.customize ['createhd', '--filename', "disk-#{i}-#{driverletters[d]}-#{DISK_UUID}.disk", '--size', $kube_node_instances_with_disks_size] # 10GB disk
vb.customize ['storageattach', :id, '--storagectl', 'SATA Controller', '--port', d, '--device', 0, '--type', 'hdd', '--medium', "disk-#{i}-#{driverletters[d]}-#{DISK_UUID}.disk", '--nonrotational', 'on', '--mtype', 'normal']
end
end
end
if $expose_docker_tcp
node.vm.network "forwarded_port", guest: 2375, host: ($expose_docker_tcp + i - 1), auto_correct: true
end
$forwarded_ports.each do |guest, host|
node.vm.network "forwarded_port", guest: guest, host: host, auto_correct: true
end
if ["rhel8"].include? $os
# Vagrant synced_folder rsync options cannot be used for RHEL boxes as Rsync package cannot
# be installed until the host is registered with a valid Red Hat support subscription
node.vm.synced_folder ".", "/vagrant", disabled: false
$shared_folders.each do |src, dst|
node.vm.synced_folder src, dst
end
else
node.vm.synced_folder ".", "/vagrant", disabled: false, type: "rsync", rsync__args: ['--verbose', '--archive', '--delete', '-z'] , rsync__exclude: ['.git','venv']
$shared_folders.each do |src, dst|
node.vm.synced_folder src, dst, type: "rsync", rsync__args: ['--verbose', '--archive', '--delete', '-z']
end
end
ip = "#{$subnet}.#{i+100}"
ip6 = "#{$subnet_ipv6}::#{i+100}"
node.vm.network :private_network,
:ip => ip,
:libvirt__guest_ipv6 => 'yes',
:libvirt__ipv6_address => ip6,
:libvirt__ipv6_prefix => "64",
:libvirt__forward_mode => "none",
:libvirt__dhcp_enabled => false
# libvirt__ipv6_address does not work as intended, the address is obtained with the desired prefix, but auto-generated(like fd3c:b398:698:756:5054:ff:fe48:c61e/64)
# add default route for detect ansible_default_ipv6
# TODO: fix libvirt__ipv6 or use $subnet in shell
config.vm.provision "shell", inline: "ip -6 r a fd3c:b398:698:756::/64 dev eth1;ip -6 r add default via fd3c:b398:0698:0756::1 dev eth1 || true"
# Disable swap for each vm
node.vm.provision "shell", inline: "swapoff -a"
# ubuntu2004 and ubuntu2204 have IPv6 explicitly disabled. This undoes that.
if ["ubuntu2004", "ubuntu2204"].include? $os
node.vm.provision "shell", inline: "rm -f /etc/modprobe.d/local.conf"
node.vm.provision "shell", inline: "sed -i '/net.ipv6.conf.all.disable_ipv6/d' /etc/sysctl.d/99-sysctl.conf /etc/sysctl.conf"
end
# Hack for fedora39/40 to get the IP address of the second interface
if ["fedora39", "fedora40", "fedora39-arm64", "fedora40-arm64"].include? $os
config.vm.provision "shell", inline: <<-SHELL
nmcli conn modify 'Wired connection 2' ipv4.addresses $(cat /etc/sysconfig/network-scripts/ifcfg-eth1 | grep IPADDR | cut -d "=" -f2)/24
nmcli conn modify 'Wired connection 2' ipv4.method manual
service NetworkManager restart
SHELL
end
# Rockylinux boxes needs UEFI
if ["rockylinux8", "rockylinux9"].include? $os
config.vm.provider "libvirt" do |domain|
domain.loader = "/usr/share/OVMF/x64/OVMF_CODE.fd"
end
end
# Disable firewalld on oraclelinux/redhat vms
if ["oraclelinux","oraclelinux8", "rhel8","rockylinux8"].include? $os
node.vm.provision "shell", inline: "systemctl stop firewalld; systemctl disable firewalld"
end
host_vars[vm_name] = {
"ip": ip,
"flannel_interface": "eth1",
"kube_network_plugin": $network_plugin,
"kube_network_plugin_multus": $multi_networking,
"download_run_once": $download_run_once,
"download_localhost": "False",
"download_cache_dir": ENV['HOME'] + "/kubespray_cache",
# Make kubespray cache even when download_run_once is false
"download_force_cache": $download_force_cache,
# Keeping the cache on the nodes can improve provisioning speed while debugging kubespray
"download_keep_remote_cache": "False",
"docker_rpm_keepcache": "1",
# These two settings will put kubectl and admin.config in $inventory/artifacts
"kubeconfig_localhost": "True",
"kubectl_localhost": "True",
"local_path_provisioner_enabled": "#{$local_path_provisioner_enabled}",
"local_path_provisioner_claim_root": "#{$local_path_provisioner_claim_root}",
"ansible_ssh_user": SUPPORTED_OS[$os][:user],
"ansible_ssh_private_key_file": File.join(Dir.home, ".vagrant.d", "insecure_private_key"),
"unsafe_show_logs": "True"
}
# Only execute the Ansible provisioner once, when all the machines are up and ready.
# And limit the action to gathering facts, the full playbook is going to be ran by testcases_run.sh
if i == $num_instances
node.vm.provision "ansible" do |ansible|
ansible.playbook = $playbook
ansible.compatibility_mode = "2.0"
ansible.verbose = $ansible_verbosity
ansible.become = true
ansible.limit = "all,localhost"
ansible.host_key_checking = false
ansible.raw_arguments = ["--forks=#{$num_instances}",
"--flush-cache",
"-e ansible_become_pass=vagrant"] +
$inventories.map {|inv| ["-i", inv]}.flatten
ansible.host_vars = host_vars
ansible.extra_vars = $extra_vars
if $ansible_tags != ""
ansible.tags = [$ansible_tags]
end
ansible.groups = {
"etcd" => ["#{$instance_name_prefix}-[#{$first_etcd}:#{$etcd_instances + $first_etcd - 1}]"],
"kube_control_plane" => ["#{$instance_name_prefix}-[#{$first_control_plane}:#{$control_plane_instances + $first_control_plane - 1}]"],
"kube_node" => ["#{$instance_name_prefix}-[#{$first_node}:#{$kube_node_instances + $first_node - 1}]"],
"k8s_cluster:children" => ["kube_control_plane", "kube_node"],
}
end
end
end
end
end
================================================
FILE: _config.yml
================================================
---
theme: jekyll-theme-slate
================================================
FILE: ansible.cfg
================================================
[ssh_connection]
pipelining=True
ssh_args = -o ControlMaster=auto -o ControlPersist=30m -o ConnectionAttempts=100 -o UserKnownHostsFile=/dev/null
#control_path = ~/.ssh/ansible-%%r@%%h:%%p
[defaults]
# https://github.com/ansible/ansible/issues/56930 (to ignore group names with - and .)
force_valid_group_names = ignore
host_key_checking=False
gathering = smart
fact_caching = jsonfile
fact_caching_connection = /tmp
fact_caching_timeout = 86400
timeout = 300
stdout_callback = default
display_skipped_hosts = no
library = ./library
callbacks_enabled = profile_tasks
roles_path = roles:$VIRTUAL_ENV/usr/local/share/kubespray/roles:$VIRTUAL_ENV/usr/local/share/ansible/roles:/usr/share/kubespray/roles
deprecation_warnings=False
inventory_ignore_extensions = ~, .orig, .bak, .ini, .cfg, .retry, .pyc, .pyo, .creds, .gpg
[inventory]
ignore_patterns = artifacts, credentials
================================================
FILE: cluster.yml
================================================
---
- name: Install Kubernetes
ansible.builtin.import_playbook: playbooks/cluster.yml
================================================
FILE: code-of-conduct.md
================================================
# Kubernetes Community Code of Conduct
Please refer to our [Kubernetes Community Code of Conduct](https://git.k8s.io/community/code-of-conduct.md)
================================================
FILE: contrib/aws_iam/kubernetes-master-policy.json
================================================
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["ec2:*"],
"Resource": ["*"]
},
{
"Effect": "Allow",
"Action": ["elasticloadbalancing:*"],
"Resource": ["*"]
},
{
"Effect": "Allow",
"Action": ["route53:*"],
"Resource": ["*"]
},
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::kubernetes-*"
]
}
]
}
================================================
FILE: contrib/aws_iam/kubernetes-master-role.json
================================================
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": { "Service": "ec2.amazonaws.com"},
"Action": "sts:AssumeRole"
}
]
}
================================================
FILE: contrib/aws_iam/kubernetes-minion-policy.json
================================================
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::kubernetes-*"
]
},
{
"Effect": "Allow",
"Action": "ec2:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "ec2:AttachVolume",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "ec2:DetachVolume",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": ["route53:*"],
"Resource": ["*"]
},
{
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:GetRepositoryPolicy",
"ecr:DescribeRepositories",
"ecr:ListImages",
"ecr:BatchGetImage"
],
"Resource": "*"
}
]
}
================================================
FILE: contrib/aws_iam/kubernetes-minion-role.json
================================================
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": { "Service": "ec2.amazonaws.com"},
"Action": "sts:AssumeRole"
}
]
}
================================================
FILE: contrib/aws_inventory/kubespray-aws-inventory.py
================================================
#!/usr/bin/env python
from __future__ import print_function
import boto3
import os
import argparse
import json
class SearchEC2Tags(object):
def __init__(self):
self.parse_args()
if self.args.list:
self.search_tags()
if self.args.host:
data = {}
print(json.dumps(data, indent=2))
def parse_args(self):
##Check if VPC_VISIBILITY is set, if not default to private
if "VPC_VISIBILITY" in os.environ:
self.vpc_visibility = os.environ['VPC_VISIBILITY']
else:
self.vpc_visibility = "private"
##Support --list and --host flags. We largely ignore the host one.
parser = argparse.ArgumentParser()
parser.add_argument('--list', action='store_true', default=False, help='List instances')
parser.add_argument('--host', action='store_true', help='Get all the variables about a specific instance')
self.args = parser.parse_args()
def search_tags(self):
hosts = {}
hosts['_meta'] = { 'hostvars': {} }
##Search ec2 three times to find nodes of each group type. Relies on kubespray-role key/value.
for group in ["kube_control_plane", "kube_node", "etcd"]:
hosts[group] = []
tag_key = "kubespray-role"
tag_value = ["*"+group+"*"]
region = os.environ['AWS_REGION']
ec2 = boto3.resource('ec2', region)
filters = [{'Name': 'tag:'+tag_key, 'Values': tag_value}, {'Name': 'instance-state-name', 'Values': ['running']}]
cluster_name = os.getenv('CLUSTER_NAME')
if cluster_name:
filters.append({'Name': 'tag-key', 'Values': ['kubernetes.io/cluster/'+cluster_name]})
instances = ec2.instances.filter(Filters=filters)
for instance in instances:
##Suppose default vpc_visibility is private
dns_name = instance.private_dns_name
ansible_host = {
'ansible_ssh_host': instance.private_ip_address
}
##Override when vpc_visibility actually is public
if self.vpc_visibility == "public":
dns_name = instance.public_dns_name
ansible_host = {
'ansible_ssh_host': instance.public_ip_address
}
##Set when instance actually has node_labels
node_labels_tag = list(filter(lambda t: t['Key'] == 'kubespray-node-labels', instance.tags))
if node_labels_tag:
ansible_host['node_labels'] = dict([ label.strip().split('=') for label in node_labels_tag[0]['Value'].split(',') ])
##Set when instance actually has node_taints
node_taints_tag = list(filter(lambda t: t['Key'] == 'kubespray-node-taints', instance.tags))
if node_taints_tag:
ansible_host['node_taints'] = list([ taint.strip() for taint in node_taints_tag[0]['Value'].split(',') ])
hosts[group].append(dns_name)
hosts['_meta']['hostvars'][dns_name] = ansible_host
hosts['k8s_cluster'] = {'children':['kube_control_plane', 'kube_node']}
print(json.dumps(hosts, sort_keys=True, indent=2))
SearchEC2Tags()
================================================
FILE: contrib/aws_inventory/requirements.txt
================================================
boto3 # Apache-2.0
================================================
FILE: contrib/azurerm/.gitignore
================================================
.generated
/inventory
================================================
FILE: contrib/azurerm/README.md
================================================
# Kubernetes on Azure with Azure Resource Group Templates
Provision the base infrastructure for a Kubernetes cluster by using [Azure Resource Group Templates](https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-authoring-templates)
## Status
This will provision the base infrastructure (vnet, vms, nics, ips, ...) needed for Kubernetes in Azure into the specified
Resource Group. It will not install Kubernetes itself, this has to be done in a later step by yourself (using kubespray of course).
## Requirements
- [Install azure-cli](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest)
- [Login with azure-cli](https://docs.microsoft.com/en-us/cli/azure/authenticate-azure-cli?view=azure-cli-latest)
- Dedicated Resource Group created in the Azure Portal or through azure-cli
## Configuration through group_vars/all
You have to modify at least two variables in group_vars/all. The one is the **cluster_name** variable, it must be globally
unique due to some restrictions in Azure. The other one is the **ssh_public_keys** variable, it must be your ssh public
key to access your azure virtual machines. Most other variables should be self explanatory if you have some basic Kubernetes
experience.
## Bastion host
You can enable the use of a Bastion Host by changing **use_bastion** in group_vars/all to **true**. The generated
templates will then include an additional bastion VM which can then be used to connect to the masters and nodes. The option
also removes all public IPs from all other VMs.
## Generating and applying
To generate and apply the templates, call:
```shell
./apply-rg.sh <resource_group_name>
```
If you change something in the configuration (e.g. number of nodes) later, you can call this again and Azure will
take care about creating/modifying whatever is needed.
## Clearing a resource group
If you need to delete all resources from a resource group, simply call:
```shell
./clear-rg.sh <resource_group_name>
```
**WARNING** this really deletes everything from your resource group, including everything that was later created by you!
## Installing Ansible and the dependencies
Install Ansible according to [Ansible installation guide](/docs/ansible/ansible.md#installing-ansible)
## Generating an inventory for kubespray
After you have applied the templates, you can generate an inventory with this call:
```shell
./generate-inventory.sh <resource_group_name>
```
It will create the file ./inventory which can then be used with kubespray, e.g.:
```shell
cd kubespray-root-dir
ansible-playbook -i contrib/azurerm/inventory -u devops --become -e "@inventory/sample/group_vars/all/all.yml" cluster.yml
```
================================================
FILE: contrib/azurerm/apply-rg.sh
================================================
#!/usr/bin/env bash
set -e
AZURE_RESOURCE_GROUP="$1"
if [ "$AZURE_RESOURCE_GROUP" == "" ]; then
echo "AZURE_RESOURCE_GROUP is missing"
exit 1
fi
ansible-playbook generate-templates.yml
az deployment group create --template-file ./.generated/network.json -g $AZURE_RESOURCE_GROUP
az deployment group create --template-file ./.generated/storage.json -g $AZURE_RESOURCE_GROUP
az deployment group create --template-file ./.generated/availability-sets.json -g $AZURE_RESOURCE_GROUP
az deployment group create --template-file ./.generated/bastion.json -g $AZURE_RESOURCE_GROUP
az deployment group create --template-file ./.generated/masters.json -g $AZURE_RESOURCE_GROUP
az deployment group create --template-file ./.generated/minions.json -g $AZURE_RESOURCE_GROUP
================================================
FILE: contrib/azurerm/clear-rg.sh
================================================
#!/usr/bin/env bash
set -e
AZURE_RESOURCE_GROUP="$1"
if [ "$AZURE_RESOURCE_GROUP" == "" ]; then
echo "AZURE_RESOURCE_GROUP is missing"
exit 1
fi
ansible-playbook generate-templates.yml
az group deployment create -g "$AZURE_RESOURCE_GROUP" --template-file ./.generated/clear-rg.json --mode Complete
================================================
FILE: contrib/azurerm/generate-inventory.sh
================================================
#!/usr/bin/env bash
set -e
AZURE_RESOURCE_GROUP="$1"
if [ "$AZURE_RESOURCE_GROUP" == "" ]; then
echo "AZURE_RESOURCE_GROUP is missing"
exit 1
fi
# check if azure cli 2.0 exists else use azure cli 1.0
if az &>/dev/null; then
ansible-playbook generate-inventory_2.yml -e azure_resource_group="$AZURE_RESOURCE_GROUP"
elif azure &>/dev/null; then
ansible-playbook generate-inventory.yml -e azure_resource_group="$AZURE_RESOURCE_GROUP"
else
echo "Azure cli not found"
fi
================================================
FILE: contrib/azurerm/generate-inventory.yml
================================================
---
- name: Generate Azure inventory
hosts: localhost
gather_facts: false
roles:
- generate-inventory
================================================
FILE: contrib/azurerm/generate-inventory_2.yml
================================================
---
- name: Generate Azure inventory
hosts: localhost
gather_facts: false
roles:
- generate-inventory_2
================================================
FILE: contrib/azurerm/generate-templates.yml
================================================
---
- name: Generate Azure templates
hosts: localhost
gather_facts: false
roles:
- generate-templates
================================================
FILE: contrib/azurerm/group_vars/all
================================================
# Due to some Azure limitations (ex:- Storage Account's name must be unique),
# this name must be globally unique - it will be used as a prefix for azure components
cluster_name: example
# Set this to true if you do not want to have public IPs for your masters and minions. This will provision a bastion
# node that can be used to access the masters and minions
use_bastion: false
# Set this to a preferred name that will be used as the first part of the dns name for your bastotion host. For example: k8s-bastion.<azureregion>.cloudapp.azure.com.
# This is convenient when exceptions have to be configured on a firewall to allow ssh to the given bastion host.
# bastion_domain_prefix: k8s-bastion
number_of_k8s_masters: 3
number_of_k8s_nodes: 3
masters_vm_size: Standard_A2
masters_os_disk_size: 1000
minions_vm_size: Standard_A2
minions_os_disk_size: 1000
admin_username: devops
admin_password: changeme
# MAKE SURE TO CHANGE THIS TO YOUR PUBLIC KEY to access your azure machines
ssh_public_keys:
- "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLRzcxbsFDdEibiyXCSdIFh7bKbXso1NqlKjEyPTptf3aBXHEhVil0lJRjGpTlpfTy7PHvXFbXIOCdv9tOmeH1uxWDDeZawgPFV6VSZ1QneCL+8bxzhjiCn8133wBSPZkN8rbFKd9eEUUBfx8ipCblYblF9FcidylwtMt5TeEmXk8yRVkPiCuEYuDplhc2H0f4PsK3pFb5aDVdaDT3VeIypnOQZZoUxHWqm6ThyHrzLJd3SrZf+RROFWW1uInIDf/SZlXojczUYoffxgT1lERfOJCHJXsqbZWugbxQBwqsVsX59+KPxFFo6nV88h3UQr63wbFx52/MXkX4WrCkAHzN ablock-vwfs@dell-lappy"
# Disable using ssh using password. Change it to false to allow to connect to ssh by password
disablePasswordAuthentication: true
# Azure CIDRs
azure_vnet_cidr: 10.0.0.0/8
azure_admin_cidr: 10.241.2.0/24
azure_masters_cidr: 10.0.4.0/24
azure_minions_cidr: 10.240.0.0/16
# Azure loadbalancer port to use to access your cluster
kube_apiserver_port: 6443
# Azure Netwoking and storage naming to use with inventory/all.yml
#azure_virtual_network_name: KubeVNET
#azure_subnet_admin_name: ad-subnet
#azure_subnet_masters_name: master-subnet
#azure_subnet_minions_name: minion-subnet
#azure_route_table_name: routetable
#azure_security_group_name: secgroup
# Storage types available are: "Standard_LRS","Premium_LRS"
#azure_storage_account_type: Standard_LRS
================================================
FILE: contrib/azurerm/roles/generate-inventory/tasks/main.yml
================================================
---
- name: Query Azure VMs
command: azure vm list-ip-address --json {{ azure_resource_group }}
register: vm_list_cmd
- name: Set vm_list
set_fact:
vm_list: "{{ vm_list_cmd.stdout }}"
- name: Generate inventory
template:
src: inventory.j2
dest: "{{ playbook_dir }}/inventory"
mode: "0644"
================================================
FILE: contrib/azurerm/roles/generate-inventory/templates/inventory.j2
================================================
{% for vm in vm_list %}
{% if not use_bastion or vm.name == 'bastion' %}
{{ vm.name }} ansible_ssh_host={{ vm.networkProfile.networkInterfaces[0].expanded.ipConfigurations[0].publicIPAddress.expanded.ipAddress }} ip={{ vm.networkProfile.networkInterfaces[0].expanded.ipConfigurations[0].privateIPAddress }}
{% else %}
{{ vm.name }} ansible_ssh_host={{ vm.networkProfile.networkInterfaces[0].expanded.ipConfigurations[0].privateIPAddress }}
{% endif %}
{% endfor %}
[kube_control_plane]
{% for vm in vm_list %}
{% if 'kube_control_plane' in vm.tags.roles %}
{{ vm.name }}
{% endif %}
{% endfor %}
[etcd]
{% for vm in vm_list %}
{% if 'etcd' in vm.tags.roles %}
{{ vm.name }}
{% endif %}
{% endfor %}
[kube_node]
{% for vm in vm_list %}
{% if 'kube_node' in vm.tags.roles %}
{{ vm.name }}
{% endif %}
{% endfor %}
[k8s_cluster:children]
kube_node
kube_control_plane
================================================
FILE: contrib/azurerm/roles/generate-inventory_2/tasks/main.yml
================================================
---
- name: Query Azure VMs IPs
command: az vm list-ip-addresses -o json --resource-group {{ azure_resource_group }}
register: vm_ip_list_cmd
- name: Query Azure VMs Roles
command: az vm list -o json --resource-group {{ azure_resource_group }}
register: vm_list_cmd
- name: Query Azure Load Balancer Public IP
command: az network public-ip show -o json -g {{ azure_resource_group }} -n kubernetes-api-pubip
register: lb_pubip_cmd
- name: Set VM IP, roles lists and load balancer public IP
set_fact:
vm_ip_list: "{{ vm_ip_list_cmd.stdout }}"
vm_roles_list: "{{ vm_list_cmd.stdout }}"
lb_pubip: "{{ lb_pubip_cmd.stdout }}"
- name: Generate inventory
template:
src: inventory.j2
dest: "{{ playbook_dir }}/inventory"
mode: "0644"
- name: Generate Load Balancer variables
template:
src: loadbalancer_vars.j2
dest: "{{ playbook_dir }}/loadbalancer_vars.yml"
mode: "0644"
================================================
FILE: contrib/azurerm/roles/generate-inventory_2/templates/inventory.j2
================================================
{% for vm in vm_ip_list %}
{% if not use_bastion or vm.virtualMachine.name == 'bastion' %}
{{ vm.virtualMachine.name }} ansible_ssh_host={{ vm.virtualMachine.network.publicIpAddresses[0].ipAddress }} ip={{ vm.virtualMachine.network.privateIpAddresses[0] }}
{% else %}
{{ vm.virtualMachine.name }} ansible_ssh_host={{ vm.virtualMachine.network.privateIpAddresses[0] }}
{% endif %}
{% endfor %}
[kube_control_plane]
{% for vm in vm_roles_list %}
{% if 'kube_control_plane' in vm.tags.roles %}
{{ vm.name }}
{% endif %}
{% endfor %}
[etcd]
{% for vm in vm_roles_list %}
{% if 'etcd' in vm.tags.roles %}
{{ vm.name }}
{% endif %}
{% endfor %}
[kube_node]
{% for vm in vm_roles_list %}
{% if 'kube_node' in vm.tags.roles %}
{{ vm.name }}
{% endif %}
{% endfor %}
[k8s_cluster:children]
kube_node
kube_control_plane
================================================
FILE: contrib/azurerm/roles/generate-inventory_2/templates/loadbalancer_vars.j2
================================================
## External LB example config
apiserver_loadbalancer_domain_name: {{ lb_pubip.dnsSettings.fqdn }}
loadbalancer_apiserver:
address: {{ lb_pubip.ipAddress }}
port: 6443
## Internal loadbalancers for apiservers
loadbalancer_apiserver_localhost: false
================================================
FILE: contrib/azurerm/roles/generate-templates/defaults/main.yml
================================================
---
apiVersion: "2015-06-15"
virtualNetworkName: "{{ azure_virtual_network_name | default('KubeVNET') }}"
subnetAdminName: "{{ azure_subnet_admin_name | default('ad-subnet') }}"
subnetMastersName: "{{ azure_subnet_masters_name | default('master-subnet') }}"
subnetMinionsName: "{{ azure_subnet_minions_name | default('minion-subnet') }}"
routeTableName: "{{ azure_route_table_name | default('routetable') }}"
securityGroupName: "{{ azure_security_group_name | default('secgroup') }}"
nameSuffix: "{{ cluster_name }}"
availabilitySetMasters: "master-avs"
availabilitySetMinions: "minion-avs"
faultDomainCount: 3
updateDomainCount: 10
bastionVmSize: Standard_A0
bastionVMName: bastion
bastionIPAddressName: bastion-pubip
disablePasswordAuthentication: true
sshKeyPath: "/home/{{ admin_username }}/.ssh/authorized_keys"
imageReference:
publisher: "OpenLogic"
offer: "CentOS"
sku: "7.5"
version: "latest"
imageReferenceJson: "{{ imageReference | to_json }}"
storageAccountName: "sa{{ nameSuffix | replace('-', '') }}"
storageAccountType: "{{ azure_storage_account_type | default('Standard_LRS') }}"
================================================
FILE: contrib/azurerm/roles/generate-templates/tasks/main.yml
================================================
---
- name: Set base_dir
set_fact:
base_dir: "{{ playbook_dir }}/.generated/"
- name: Create base_dir
file:
path: "{{ base_dir }}"
state: directory
recurse: true
mode: "0755"
- name: Store json files in base_dir
template:
src: "{{ item }}"
dest: "{{ base_dir }}/{{ item }}"
mode: "0644"
with_items:
- network.json
- storage.json
- availability-sets.json
- bastion.json
- masters.json
- minions.json
- clear-rg.json
================================================
FILE: contrib/azurerm/roles/generate-templates/templates/availability-sets.json
================================================
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
},
"resources": [
{
"type": "Microsoft.Compute/availabilitySets",
"name": "{{availabilitySetMasters}}",
"apiVersion": "{{apiVersion}}",
"location": "[resourceGroup().location]",
"properties": {
"PlatformFaultDomainCount": "{{faultDomainCount}}",
"PlatformUpdateDomainCount": "{{updateDomainCount}}"
}
},
{
"type": "Microsoft.Compute/availabilitySets",
"name": "{{availabilitySetMinions}}",
"apiVersion": "{{apiVersion}}",
"location": "[resourceGroup().location]",
"properties": {
"PlatformFaultDomainCount": "{{faultDomainCount}}",
"PlatformUpdateDomainCount": "{{updateDomainCount}}"
}
}
]
}
================================================
FILE: contrib/azurerm/roles/generate-templates/templates/bastion.json
================================================
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"vnetID": "[resourceId('Microsoft.Network/virtualNetworks', '{{virtualNetworkName}}')]",
"subnetAdminRef": "[concat(variables('vnetID'),'/subnets/', '{{subnetAdminName}}')]"
},
"resources": [
{% if use_bastion %}
{
"apiVersion": "{{apiVersion}}",
"type": "Microsoft.Network/publicIPAddresses",
"name": "{{bastionIPAddressName}}",
"location": "[resourceGroup().location]",
"properties": {
"publicIPAllocationMethod": "Static",
"dnsSettings": {
{% if bastion_domain_prefix %}
"domainNameLabel": "{{ bastion_domain_prefix }}"
{% endif %}
}
}
},
{
"apiVersion": "{{apiVersion}}",
"type": "Microsoft.Network/networkInterfaces",
"name": "{{bastionVMName}}-nic",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/', '{{bastionIPAddressName}}')]"
],
"properties": {
"ipConfigurations": [
{
"name": "BastionIpConfig",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses', '{{bastionIPAddressName}}')]"
},
"subnet": {
"id": "[variables('subnetAdminRef')]"
}
}
}
]
}
},
{
"apiVersion": "{{apiVersion}}",
"type": "Microsoft.Compute/virtualMachines",
"name": "{{bastionVMName}}",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.Network/networkInterfaces/', '{{bastionVMName}}-nic')]"
],
"tags": {
"roles": "bastion"
},
"properties": {
"hardwareProfile": {
"vmSize": "{{bastionVmSize}}"
},
"osProfile": {
"computerName": "{{bastionVMName}}",
"adminUsername": "{{admin_username}}",
"adminPassword": "{{admin_password}}",
"linuxConfiguration": {
"disablePasswordAuthentication": "true",
"ssh": {
"publicKeys": [
{% for key in ssh_public_keys %}
{
"path": "{{sshKeyPath}}",
"keyData": "{{key}}"
}{% if loop.index < ssh_public_keys | length %},{% endif %}
{% endfor %}
]
}
}
},
"storageProfile": {
"imageReference": {{imageReferenceJson}},
"osDisk": {
"name": "osdisk",
"vhd": {
"uri": "[concat('http://', '{{storageAccountName}}', '.blob.core.windows.net/vhds/', '{{bastionVMName}}', '-osdisk.vhd')]"
},
"caching": "ReadWrite",
"createOption": "FromImage"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', '{{bastionVMName}}-nic')]"
}
]
}
}
}
{% endif %}
]
}
================================================
FILE: contrib/azurerm/roles/generate-templates/templates/clear-rg.json
================================================
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [],
"outputs": {}
}
================================================
FILE: contrib/azurerm/roles/generate-templates/templates/masters.json
================================================
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"lbDomainName": "{{nameSuffix}}-api",
"lbPublicIPAddressName": "kubernetes-api-pubip",
"lbPublicIPAddressType": "Static",
"lbPublicIPAddressID": "[resourceId('Microsoft.Network/publicIPAddresses',variables('lbPublicIPAddressName'))]",
"lbName": "kubernetes-api",
"lbID": "[resourceId('Microsoft.Network/loadBalancers',variables('lbName'))]",
"vnetID": "[resourceId('Microsoft.Network/virtualNetworks', '{{virtualNetworkName}}')]",
"kubeMastersSubnetRef": "[concat(variables('vnetID'),'/subnets/', '{{subnetMastersName}}')]"
},
"resources": [
{
"apiVersion": "{{apiVersion}}",
"type": "Microsoft.Network/publicIPAddresses",
"name": "[variables('lbPublicIPAddressName')]",
"location": "[resourceGroup().location]",
"properties": {
"publicIPAllocationMethod": "[variables('lbPublicIPAddressType')]",
"dnsSettings": {
"domainNameLabel": "[variables('lbDomainName')]"
}
}
},
{
"apiVersion": "{{apiVersion}}",
"name": "[variables('lbName')]",
"type": "Microsoft.Network/loadBalancers",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/', variables('lbPublicIPAddressName'))]"
],
"properties": {
"frontendIPConfigurations": [
{
"name": "kube-api-frontend",
"properties": {
"publicIPAddress": {
"id": "[variables('lbPublicIPAddressID')]"
}
}
}
],
"backendAddressPools": [
{
"name": "kube-api-backend"
}
],
"loadBalancingRules": [
{
"name": "kube-api",
"properties": {
"frontendIPConfiguration": {
"id": "[concat(variables('lbID'), '/frontendIPConfigurations/kube-api-frontend')]"
},
"backendAddressPool": {
"id": "[concat(variables('lbID'), '/backendAddressPools/kube-api-backend')]"
},
"protocol": "tcp",
"frontendPort": "{{kube_apiserver_port}}",
"backendPort": "{{kube_apiserver_port}}",
"enableFloatingIP": false,
"idleTimeoutInMinutes": 5,
"probe": {
"id": "[concat(variables('lbID'), '/probes/kube-api')]"
}
}
}
],
"probes": [
{
"name": "kube-api",
"properties": {
"protocol": "tcp",
"port": "{{kube_apiserver_port}}",
"intervalInSeconds": 5,
"numberOfProbes": 2
}
}
]
}
},
{% for i in range(number_of_k8s_masters) %}
{% if not use_bastion %}
{
"apiVersion": "{{apiVersion}}",
"type": "Microsoft.Network/publicIPAddresses",
"name": "master-{{i}}-pubip",
"location": "[resourceGroup().location]",
"properties": {
"publicIPAllocationMethod": "Static"
}
},
{% endif %}
{
"apiVersion": "{{apiVersion}}",
"type": "Microsoft.Network/networkInterfaces",
"name": "master-{{i}}-nic",
"location": "[resourceGroup().location]",
"dependsOn": [
{% if not use_bastion %}
"[concat('Microsoft.Network/publicIPAddresses/', 'master-{{i}}-pubip')]",
{% endif %}
"[concat('Microsoft.Network/loadBalancers/', variables('lbName'))]"
],
"properties": {
"ipConfigurations": [
{
"name": "MastersIpConfig",
"properties": {
"privateIPAllocationMethod": "Dynamic",
{% if not use_bastion %}
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses', 'master-{{i}}-pubip')]"
},
{% endif %}
"subnet": {
"id": "[variables('kubeMastersSubnetRef')]"
},
"loadBalancerBackendAddressPools": [
{
"id": "[concat(variables('lbID'), '/backendAddressPools/kube-api-backend')]"
}
]
}
}
],
"networkSecurityGroup": {
"id": "[resourceId('Microsoft.Network/networkSecurityGroups', '{{securityGroupName}}')]"
},
"enableIPForwarding": true
}
},
{
"type": "Microsoft.Compute/virtualMachines",
"name": "master-{{i}}",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.Network/networkInterfaces/', 'master-{{i}}-nic')]"
],
"tags": {
"roles": "kube_control_plane,etcd"
},
"apiVersion": "{{apiVersion}}",
"properties": {
"availabilitySet": {
"id": "[resourceId('Microsoft.Compute/availabilitySets', '{{availabilitySetMasters}}')]"
},
"hardwareProfile": {
"vmSize": "{{masters_vm_size}}"
},
"osProfile": {
"computerName": "master-{{i}}",
"adminUsername": "{{admin_username}}",
"adminPassword": "{{admin_password}}",
"linuxConfiguration": {
"disablePasswordAuthentication": "{{disablePasswordAuthentication}}",
"ssh": {
"publicKeys": [
{% for key in ssh_public_keys %}
{
"path": "{{sshKeyPath}}",
"keyData": "{{key}}"
}{% if loop.index < ssh_public_keys | length %},{% endif %}
{% endfor %}
]
}
}
},
"storageProfile": {
"imageReference": {{imageReferenceJson}},
"osDisk": {
"name": "ma{{nameSuffix}}{{i}}",
"vhd": {
"uri": "[concat('http://','{{storageAccountName}}','.blob.core.windows.net/vhds/master-{{i}}.vhd')]"
},
"caching": "ReadWrite",
"createOption": "FromImage",
"diskSizeGB": "{{masters_os_disk_size}}"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', 'master-{{i}}-nic')]"
}
]
}
}
} {% if not loop.last %},{% endif %}
{% endfor %}
]
}
================================================
FILE: contrib/azurerm/roles/generate-templates/templates/minions.json
================================================
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"vnetID": "[resourceId('Microsoft.Network/virtualNetworks', '{{virtualNetworkName}}')]",
"kubeMinionsSubnetRef": "[concat(variables('vnetID'),'/subnets/', '{{subnetMinionsName}}')]"
},
"resources": [
{% for i in range(number_of_k8s_nodes) %}
{% if not use_bastion %}
{
"apiVersion": "{{apiVersion}}",
"type": "Microsoft.Network/publicIPAddresses",
"name": "minion-{{i}}-pubip",
"location": "[resourceGroup().location]",
"properties": {
"publicIPAllocationMethod": "Static"
}
},
{% endif %}
{
"apiVersion": "{{apiVersion}}",
"type": "Microsoft.Network/networkInterfaces",
"name": "minion-{{i}}-nic",
"location": "[resourceGroup().location]",
"dependsOn": [
{% if not use_bastion %}
"[concat('Microsoft.Network/publicIPAddresses/', 'minion-{{i}}-pubip')]"
{% endif %}
],
"properties": {
"ipConfigurations": [
{
"name": "MinionsIpConfig",
"properties": {
"privateIPAllocationMethod": "Dynamic",
{% if not use_bastion %}
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses', 'minion-{{i}}-pubip')]"
},
{% endif %}
"subnet": {
"id": "[variables('kubeMinionsSubnetRef')]"
}
}
}
],
"networkSecurityGroup": {
"id": "[resourceId('Microsoft.Network/networkSecurityGroups', '{{securityGroupName}}')]"
},
"enableIPForwarding": true
}
},
{
"type": "Microsoft.Compute/virtualMachines",
"name": "minion-{{i}}",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.Network/networkInterfaces/', 'minion-{{i}}-nic')]"
],
"tags": {
"roles": "kube_node"
},
"apiVersion": "{{apiVersion}}",
"properties": {
"availabilitySet": {
"id": "[resourceId('Microsoft.Compute/availabilitySets', '{{availabilitySetMinions}}')]"
},
"hardwareProfile": {
"vmSize": "{{minions_vm_size}}"
},
"osProfile": {
"computerName": "minion-{{i}}",
"adminUsername": "{{admin_username}}",
"adminPassword": "{{admin_password}}",
"linuxConfiguration": {
"disablePasswordAuthentication": "{{disablePasswordAuthentication}}",
"ssh": {
"publicKeys": [
{% for key in ssh_public_keys %}
{
"path": "{{sshKeyPath}}",
"keyData": "{{key}}"
}{% if loop.index < ssh_public_keys | length %},{% endif %}
{% endfor %}
]
}
}
},
"storageProfile": {
"imageReference": {{imageReferenceJson}},
"osDisk": {
"name": "mi{{nameSuffix}}{{i}}",
"vhd": {
"uri": "[concat('http://','{{storageAccountName}}','.blob.core.windows.net/vhds/minion-{{i}}.vhd')]"
},
"caching": "ReadWrite",
"createOption": "FromImage",
"diskSizeGB": "{{minions_os_disk_size}}"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', 'minion-{{i}}-nic')]"
}
]
}
}
} {% if not loop.last %},{% endif %}
{% endfor %}
]
}
================================================
FILE: contrib/azurerm/roles/generate-templates/templates/network.json
================================================
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
},
"resources": [
{
"apiVersion": "{{apiVersion}}",
"type": "Microsoft.Network/routeTables",
"name": "{{routeTableName}}",
"location": "[resourceGroup().location]",
"properties": {
"routes": [
]
}
},
{
"type": "Microsoft.Network/virtualNetworks",
"name": "{{virtualNetworkName}}",
"location": "[resourceGroup().location]",
"apiVersion": "{{apiVersion}}",
"dependsOn": [
"[concat('Microsoft.Network/routeTables/', '{{routeTableName}}')]"
],
"properties": {
"addressSpace": {
"addressPrefixes": [
"{{azure_vnet_cidr}}"
]
},
"subnets": [
{
"name": "{{subnetMastersName}}",
"properties": {
"addressPrefix": "{{azure_masters_cidr}}",
"routeTable": {
"id": "[resourceId('Microsoft.Network/routeTables', '{{routeTableName}}')]"
}
}
},
{
"name": "{{subnetMinionsName}}",
"properties": {
"addressPrefix": "{{azure_minions_cidr}}",
"routeTable": {
"id": "[resourceId('Microsoft.Network/routeTables', '{{routeTableName}}')]"
}
}
}
{% if use_bastion %}
,{
"name": "{{subnetAdminName}}",
"properties": {
"addressPrefix": "{{azure_admin_cidr}}",
"routeTable": {
"id": "[resourceId('Microsoft.Network/routeTables', '{{routeTableName}}')]"
}
}
}
{% endif %}
]
}
},
{
"apiVersion": "{{apiVersion}}",
"type": "Microsoft.Network/networkSecurityGroups",
"name": "{{securityGroupName}}",
"location": "[resourceGroup().location]",
"properties": {
"securityRules": [
{% if not use_bastion %}
{
"name": "ssh",
"properties": {
"description": "Allow SSH",
"protocol": "Tcp",
"sourcePortRange": "*",
"destinationPortRange": "22",
"sourceAddressPrefix": "Internet",
"destinationAddressPrefix": "*",
"access": "Allow",
"priority": 100,
"direction": "Inbound"
}
},
{% endif %}
{
"name": "kube-api",
"properties": {
"description": "Allow secure kube-api",
"protocol": "Tcp",
"sourcePortRange": "*",
"destinationPortRange": "{{kube_apiserver_port}}",
"sourceAddressPrefix": "Internet",
"destinationAddressPrefix": "*",
"access": "Allow",
"priority": 101,
"direction": "Inbound"
}
}
]
},
"resources": [],
"dependsOn": []
}
]
}
================================================
FILE: contrib/azurerm/roles/generate-templates/templates/storage.json
================================================
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"name": "{{storageAccountName}}",
"location": "[resourceGroup().location]",
"apiVersion": "{{apiVersion}}",
"properties": {
"accountType": "{{storageAccountType}}"
}
}
]
}
================================================
FILE: contrib/collection.sh
================================================
#!/bin/bash -eux
# Install collection from source assuming dependencies are present.
# Run in SemaphoreUI this bash script can install Kubespray from the repo
NAMESPACE=kubernetes_sigs
COLLECTION=kubespray
MY_VER=$(grep '^version:' galaxy.yml|cut -d: -f2|sed 's/ //')
ansible-galaxy collection build --force --output-path .
ansible-galaxy collection install --offline --force $NAMESPACE-$COLLECTION-$MY_VER.tar.gz
================================================
FILE: contrib/offline/README.md
================================================
# Offline deployment
## manage-offline-container-images.sh
Container image collecting script for offline deployment
This script has two features:
(1) Get container images from an environment which is deployed online, or set IMAGES_FROM_FILE
environment variable to get images from a file (e.g. temp/images.list after running the
./generate_list.sh script).
(2) Deploy local container registry and register the container images to the registry.
Step(1) should be done online site as a preparation, then we bring the gotten images
to the target offline environment. if images are from a private registry,
you need to set `PRIVATE_REGISTRY` environment variable.
Then we will run step(2) for registering the images to local registry, or to an existing
registry set by the `DESTINATION_REGISTRY` environment variable. By default, the local registry
will run on port 5000. This can be changed with the `REGISTRY_PORT` environment variable
Step(1) can be operated with:
```shell
manage-offline-container-images.sh create
```
Step(2) can be operated with:
```shell
manage-offline-container-images.sh register
```
## generate_list.sh
This script generates the list of downloaded files and the list of container images by `roles/kubespray_defaults/defaults/main/download.yml` file.
Run this script will execute `generate_list.yml` playbook in kubespray root directory and generate four files,
all downloaded files url in files.list, all container images in images.list, jinja2 templates in *.template.
```shell
./generate_list.sh
tree temp
temp
├── files.list
├── files.list.template
├── images.list
└── images.list.template
0 directories, 5 files
```
In some cases you may want to update some component version, you can declare version variables in ansible inventory file or group_vars,
then run `./generate_list.sh -i [inventory_file]` to update file.list and images.list.
## manage-offline-files.sh
This script will download all files according to `temp/files.list` and run nginx container to provide offline file download.
Step(1) generate `files.list`
```shell
./generate_list.sh
```
Step(2) download files and run nginx container
```shell
./manage-offline-files.sh
```
when nginx container is running, it can be accessed through <http://127.0.0.1:8080/>.
## upload2artifactory.py
After the steps above, this script can recursively upload each file under a directory to a generic repository in Artifactory.
Environment Variables:
- USERNAME -- At least permissions'Deploy/Cache' and 'Delete/Overwrite'.
- TOKEN -- Generate this with 'Set Me Up' in your user.
- BASE_URL -- The URL including the repository name.
Step(3) (optional) upload files to Artifactory
```shell
cd kubespray/contrib/offline/offline-files
export USERNAME=admin
export TOKEN=...
export BASE_URL=https://artifactory.example.com/artifactory/a-generic-repo/
./upload2artifactory.py
```
================================================
FILE: contrib/offline/docker-daemon.json
================================================
{ "insecure-registries":["HOSTNAME:5000"] }
================================================
FILE: contrib/offline/generate_list.sh
================================================
#!/bin/bash
set -eo pipefail
CURRENT_DIR=$(cd $(dirname $0); pwd)
TEMP_DIR="${CURRENT_DIR}/temp"
REPO_ROOT_DIR="${CURRENT_DIR%/contrib/offline}"
: ${DOWNLOAD_YML:="roles/kubespray_defaults/defaults/main/download.yml"}
mkdir -p ${TEMP_DIR}
# generate all download files url template
grep 'download_url:' ${REPO_ROOT_DIR}/${DOWNLOAD_YML} \
| sed 's/^.*_url: //g;s/\"//g' > ${TEMP_DIR}/files.list.template
# generate all images list template
sed -n '/^downloads:/,/download_defaults:/p' ${REPO_ROOT_DIR}/${DOWNLOAD_YML} \
| sed -n "s/repo: //p;s/tag: //p" | tr -d ' ' \
| sed 'N;s#\n# #g' | tr ' ' ':' | sed 's/\"//g' > ${TEMP_DIR}/images.list.template
# add kube-* images to images list template
# Those container images are downloaded by kubeadm, then roles/kubespray_defaults/defaults/main/download.yml
# doesn't contain those images. That is reason why here needs to put those images into the
# list separately.
KUBE_IMAGES="kube-apiserver kube-controller-manager kube-scheduler kube-proxy"
for i in $KUBE_IMAGES; do
echo "{{ kube_image_repo }}/$i:v{{ kube_version }}" >> ${TEMP_DIR}/images.list.template
done
# run ansible to expand templates
/bin/cp ${CURRENT_DIR}/generate_list.yml ${REPO_ROOT_DIR}
(cd ${REPO_ROOT_DIR} && ansible-playbook $* generate_list.yml && /bin/rm generate_list.yml) || exit 1
================================================
FILE: contrib/offline/generate_list.yml
================================================
---
- name: Collect container images for offline deployment
hosts: localhost
become: false
roles:
# Just load default variables from roles.
- role: kubespray_defaults
when: false
- role: download
when: false
tasks:
# Generate files.list and images.list files from templates.
- name: Collect container images for offline deployment
template:
src: ./contrib/offline/temp/{{ item }}.list.template
dest: ./contrib/offline/temp/{{ item }}.list
mode: "0644"
with_items:
- files
- images
================================================
FILE: contrib/offline/manage-offline-container-images.sh
================================================
#!/usr/bin/env bash
OPTION=$1
CURRENT_DIR=$(cd $(dirname $0); pwd)
TEMP_DIR="${CURRENT_DIR}/temp"
IMAGE_TAR_FILE="${CURRENT_DIR}/container-images.tar.gz"
IMAGE_DIR="${CURRENT_DIR}/container-images"
IMAGE_LIST="${IMAGE_DIR}/container-images.txt"
RETRY_COUNT=5
function create_container_image_tar() {
set -e
if [ -z "${IMAGES_FROM_FILE}" ]; then
echo "Getting images from current \"$(kubectl config current-context)\""
IMAGES=$(mktemp --suffix=-images)
trap 'rm -f "${IMAGES}"' EXIT
kubectl describe cronjobs,jobs,pods --all-namespaces | grep " Image:" | awk '{print $2}' | sort | uniq > "${IMAGES}"
# NOTE: etcd and pause cannot be seen as pods.
kubectl cluster-info dump | grep -E "quay.io/coreos/etcd:|registry.k8s.io/pause:" | sed s@\"@@g >> "${IMAGES}"
else
echo "Getting images from file \"${IMAGES_FROM_FILE}\""
if [ ! -f "${IMAGES_FROM_FILE}" ]; then
echo "${IMAGES_FROM_FILE} is not a file"
exit 1
fi
IMAGES=$(realpath $IMAGES_FROM_FILE)
fi
rm -f ${IMAGE_TAR_FILE}
rm -rf ${IMAGE_DIR}
mkdir ${IMAGE_DIR}
cd ${IMAGE_DIR}
sudo --preserve-env=http_proxy,https_proxy,no_proxy ${runtime} pull registry:latest
sudo ${runtime} save -o registry-latest.tar registry:latest
while read -r image
do
FILE_NAME="$(echo ${image} | sed s@"/"@"-"@g | sed s/":"/"-"/g | sed -E 's/\@.*//g')".tar
set +e
for step in $(seq 1 ${RETRY_COUNT})
do
sudo --preserve-env=http_proxy,https_proxy,no_proxy ${runtime} pull ${image}
if [ $? -eq 0 ]; then
break
fi
echo "Failed to pull ${image} at step ${step}"
if [ ${step} -eq ${RETRY_COUNT} ]; then
exit 1
fi
done
set -e
sudo ${runtime} save -o ${FILE_NAME} ${image}
# NOTE: Here removes the following repo parts from each image
# so that these parts will be replaced with Kubespray.
# - kube_image_repo: "registry.k8s.io"
# - gcr_image_repo: "gcr.io"
# - ghcr_image_repo: "ghcr.io"
# - docker_image_repo: "docker.io"
# - quay_image_repo: "quay.io"
FIRST_PART=$(echo ${image} | awk -F"/" '{print $1}')
if [ "${FIRST_PART}" = "registry.k8s.io" ] ||
[ "${FIRST_PART}" = "gcr.io" ] ||
[ "${FIRST_PART}" = "ghcr.io" ] ||
[ "${FIRST_PART}" = "docker.io" ] ||
[ "${FIRST_PART}" = "quay.io" ] ||
[ "${FIRST_PART}" = "${PRIVATE_REGISTRY}" ]; then
image=$(echo ${image} | sed s@"${FIRST_PART}/"@@ | sed -E 's/\@.*/\n/g')
fi
echo "${FILE_NAME} ${image}" >> ${IMAGE_LIST}
done < "${IMAGES}"
cd ..
sudo chown ${USER} ${IMAGE_DIR}/*
tar -zcvf ${IMAGE_TAR_FILE} ./container-images
rm -rf ${IMAGE_DIR}
echo ""
echo "${IMAGE_TAR_FILE} is created to contain your container images."
echo "Please keep this file and bring it to your offline environment."
}
function register_container_images() {
create_registry=false
REGISTRY_PORT=${REGISTRY_PORT:-"5000"}
if [ -z "${DESTINATION_REGISTRY}" ]; then
echo "DESTINATION_REGISTRY not set, will create local registry"
create_registry=true
DESTINATION_REGISTRY="$(hostname):${REGISTRY_PORT}"
fi
echo "Images will be pushed to ${DESTINATION_REGISTRY}"
if [ ! -f ${IMAGE_TAR_FILE} ]; then
echo "${IMAGE_TAR_FILE} should exist."
exit 1
fi
if [ ! -d ${TEMP_DIR} ]; then
mkdir ${TEMP_DIR}
fi
# To avoid "http: server gave http response to https client" error.
if [ -d /etc/docker/ ]; then
set -e
# Ubuntu18.04, RHEL7/CentOS7
cp ${CURRENT_DIR}/docker-daemon.json ${TEMP_DIR}/docker-daemon.json
sed -i s@"HOSTNAME"@"$(hostname)"@ ${TEMP_DIR}/docker-daemon.json
sudo cp ${TEMP_DIR}/docker-daemon.json /etc/docker/daemon.json
elif [ -d /etc/containers/ ]; then
set -e
# RHEL8/CentOS8
cp ${CURRENT_DIR}/registries.conf ${TEMP_DIR}/registries.conf
sed -i s@"HOSTNAME"@"$(hostname)"@ ${TEMP_DIR}/registries.conf
sudo cp ${TEMP_DIR}/registries.conf /etc/containers/registries.conf
elif [ "$(uname)" == "Darwin" ]; then
echo "This is a Mac, no configuration changes are required"
else
echo "runtime package(docker-ce, podman, nerctl, etc.) should be installed"
exit 1
fi
tar -zxvf ${IMAGE_TAR_FILE}
if ${create_registry}; then
sudo ${runtime} load -i ${IMAGE_DIR}/registry-latest.tar
set +e
sudo ${runtime} container inspect registry >/dev/null 2>&1
if [ $? -ne 0 ]; then
sudo ${runtime} run --restart=always -d -p "${REGISTRY_PORT}":"${REGISTRY_PORT}" --name registry registry:latest
fi
set -e
fi
while read -r line; do
file_name=$(echo ${line} | awk '{print $1}')
raw_image=$(echo ${line} | awk '{print $2}')
new_image="${DESTINATION_REGISTRY}/${raw_image}"
load_image=$(sudo ${runtime} load -i ${IMAGE_DIR}/${file_name} | head -n1)
org_image=$(echo "${load_image}" | awk '{print $3}')
# special case for tags containing the digest when using docker or podman as the container runtime
if [ "${org_image}" == "ID:" ]; then
org_image=$(echo "${load_image}" | awk '{print $4}')
fi
image_id=$(sudo ${runtime} image inspect --format "{{.Id}}" "${org_image}")
if [ -z "${file_name}" ]; then
echo "Failed to get file_name for line ${line}"
exit 1
fi
if [ -z "${raw_image}" ]; then
echo "Failed to get raw_image for line ${line}"
exit 1
fi
if [ -z "${org_image}" ]; then
echo "Failed to get org_image for line ${line}"
exit 1
fi
if [ -z "${image_id}" ]; then
echo "Failed to get image_id for file ${file_name}"
exit 1
fi
sudo ${runtime} load -i ${IMAGE_DIR}/${file_name}
sudo ${runtime} tag ${image_id} ${new_image}
sudo ${runtime} push ${new_image}
done <<< "$(cat ${IMAGE_LIST})"
echo "Succeeded to register container images to local registry."
echo "Please specify \"${DESTINATION_REGISTRY}\" for the following options in your inventry:"
echo "- kube_image_repo"
echo "- gcr_image_repo"
echo "- docker_image_repo"
echo "- quay_image_repo"
}
# get runtime command
if command -v nerdctl 1>/dev/null 2>&1; then
runtime="nerdctl"
elif command -v podman 1>/dev/null 2>&1; then
runtime="podman"
elif command -v docker 1>/dev/null 2>&1; then
runtime="docker"
else
echo "No supported container runtime found"
exit 1
fi
if [ "${OPTION}" == "create" ]; then
create_container_image_tar
elif [ "${OPTION}" == "register" ]; then
register_container_images
else
echo "This script has two features:"
echo "(1) Get container images from an environment which is deployed online, or set IMAGES_FROM_FILE"
echo " environment variable to get images from a file (e.g. temp/images.list after running the"
echo " ./generate_list.sh script)."
echo "(2) Deploy local container registry and register the container images to the registry."
echo ""
echo "Step(1) should be done online site as a preparation, then we bring"
echo "the gotten images to the target offline environment. if images are from"
echo "a private registry, you need to set PRIVATE_REGISTRY environment variable."
echo "Then we will run step(2) for registering the images to local registry, or to an existing"
echo "registry set by the DESTINATION_REGISTRY environment variable. By default, the local registry"
echo "will run on port 5000. This can be changed with the REGISTRY_PORT environment variable"
echo ""
echo "${IMAGE_TAR_FILE} is created to contain your container images."
echo "Please keep this file and bring it to your offline environment."
echo ""
echo "Step(1) can be operated with:"
echo " $ ./manage-offline-container-images.sh create"
echo ""
echo "Step(2) can be operated with:"
echo " $ ./manage-offline-container-images.sh register"
echo ""
echo "Please specify 'create' or 'register'."
echo ""
exit 1
fi
================================================
FILE: contrib/offline/manage-offline-files.sh
================================================
#!/bin/bash
CURRENT_DIR=$( dirname "$(readlink -f "$0")" )
OFFLINE_FILES_DIR_NAME="offline-files"
OFFLINE_FILES_DIR="${CURRENT_DIR}/${OFFLINE_FILES_DIR_NAME}"
OFFLINE_FILES_ARCHIVE="${CURRENT_DIR}/offline-files.tar.gz"
FILES_LIST=${FILES_LIST:-"${CURRENT_DIR}/temp/files.list"}
NGINX_PORT=8080
# download files
if [ ! -f "${FILES_LIST}" ]; then
echo "${FILES_LIST} should exist, run ./generate_list.sh first."
exit 1
fi
rm -rf "${OFFLINE_FILES_DIR}"
rm "${OFFLINE_FILES_ARCHIVE}"
mkdir "${OFFLINE_FILES_DIR}"
while read -r url; do
if ! wget -x -P "${OFFLINE_FILES_DIR}" "${url}"; then
exit 1
fi
done < "${FILES_LIST}"
tar -czvf "${OFFLINE_FILES_ARCHIVE}" "${OFFLINE_FILES_DIR_NAME}"
[ -n "$NO_HTTP_SERVER" ] && echo "skip to run nginx" && exit 0
# run nginx container server
if command -v nerdctl 1>/dev/null 2>&1; then
runtime="nerdctl"
elif command -v podman 1>/dev/null 2>&1; then
runtime="podman"
elif command -v docker 1>/dev/null 2>&1; then
runtime="docker"
else
echo "No supported container runtime found"
exit 1
fi
sudo "${runtime}" container inspect nginx >/dev/null 2>&1
if [ $? -ne 0 ]; then
sudo --preserve-env=http_proxy,https_proxy,no_proxy "${runtime}" run \
--restart=always -d -p ${NGINX_PORT}:80 \
--volume "${OFFLINE_FILES_DIR}":/usr/share/nginx/html/download \
--volume "${CURRENT_DIR}"/nginx.conf:/etc/nginx/nginx.conf \
--name nginx nginx:alpine
fi
================================================
FILE: contrib/offline/nginx.conf
================================================
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
default_type application/octet-stream;
include /etc/nginx/conf.d/*.conf;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
include /etc/nginx/default.d/*.conf;
location / {
root /usr/share/nginx/html/download;
autoindex on;
autoindex_exact_size off;
autoindex_localtime on;
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
}
================================================
FILE: contrib/offline/registries.conf
================================================
[registries.search]
registries = ['registry.access.redhat.com', 'registry.redhat.io', 'docker.io']
[registries.insecure]
registries = ['HOSTNAME:5000']
[registries.block]
registries = []
================================================
FILE: contrib/offline/upload2artifactory.py
================================================
#!/usr/bin/env python3
"""This is a helper script to manage-offline-files.sh.
After running manage-offline-files.sh, you can run upload2artifactory.py
to recursively upload each file to a generic repository in Artifactory.
This script recurses the current working directory and is intended to
be started from 'kubespray/contrib/offline/offline-files'
Environment Variables:
USERNAME -- At least permissions'Deploy/Cache' and 'Delete/Overwrite'.
TOKEN -- Generate this with 'Set Me Up' in your user.
BASE_URL -- The URL including the repository name.
"""
import os
import urllib.request
import base64
def upload_file(file_path, destination_url, username, token):
"""Helper function to upload a single file"""
try:
with open(file_path, 'rb') as f:
file_data = f.read()
request = urllib.request.Request(destination_url, data=file_data, method='PUT') # NOQA
auth_header = base64.b64encode(f"{username}:{token}".encode()).decode()
request.add_header("Authorization", f"Basic {auth_header}")
with urllib.request.urlopen(request) as response:
if response.status in [200, 201]:
print(f"Success: Uploaded {file_path}")
else:
print(f"Failed: {response.status} {response.read().decode('utf-8')}") # NOQA
except urllib.error.HTTPError as e:
print(f"HTTPError: {e.code} {e.reason} for {file_path}")
except urllib.error.URLError as e:
print(f"URLError: {e.reason} for {file_path}")
except OSError as e:
print(f"OSError: {e.strerror} for {file_path}")
def upload_files(base_url, username, token):
""" Recurse current dir and upload each file using urllib.request """
for root, _, files in os.walk(os.getcwd()):
for file in files:
file_path = os.path.join(root, file)
relative_path = os.path.relpath(file_path, os.getcwd())
destination_url = f"{base_url}/{relative_path}"
print(f"Uploading {file_path} to {destination_url}")
upload_file(file_path, destination_url, username, token)
if __name__ == "__main__":
a_user = os.getenv("USERNAME")
a_token = os.getenv("TOKEN")
a_url = os.getenv("BASE_URL")
if not a_user or not a_token or not a_url:
print(
"Error: Environment variables USERNAME, TOKEN, and BASE_URL must be set." # NOQA
)
exit()
upload_files(a_url, a_user, a_token)
================================================
FILE: contrib/os-services/os-services.yml
================================================
---
- name: Disable firewalld/ufw
hosts: all
roles:
- { role: prepare }
================================================
FILE: contrib/os-services/roles/prepare/defaults/main.yml
================================================
---
disable_service_firewall: false
================================================
FILE: contrib/os-services/roles/prepare/tasks/main.yml
================================================
---
- name: Disable firewalld and ufw
when:
- disable_service_firewall is defined and disable_service_firewall
block:
- name: List services
service_facts:
- name: Disable service firewalld
systemd_service:
name: firewalld
state: stopped
enabled: false
when:
"'firewalld.service' in services and services['firewalld.service'].status != 'not-found'"
- name: Disable service ufw
systemd_service:
name: ufw
state: stopped
enabled: false
when:
"'ufw.service' in services and services['ufw.service'].status != 'not-found'"
================================================
FILE: contrib/terraform/aws/.gitignore
================================================
*.tfstate*
.terraform.lock.hcl
.terraform
================================================
FILE: contrib/terraform/aws/README.md
================================================
# Kubernetes on AWS with Terraform
## Overview
This project will create:
- VPC with Public and Private Subnets in # Availability Zones
- Bastion Hosts and NAT Gateways in the Public Subnet
- A dynamic number of masters, etcd, and worker nodes in the Private Subnet
- even distributed over the # of Availability Zones
- AWS ELB in the Public Subnet for accessing the Kubernetes API from the internet
## Requirements
- Terraform 0.12.0 or newer
## How to Use
- Export the variables for your AWS credentials or edit `credentials.tfvars`:
```commandline
export TF_VAR_AWS_ACCESS_KEY_ID="www"
export TF_VAR_AWS_SECRET_ACCESS_KEY ="xxx"
export TF_VAR_AWS_SSH_KEY_NAME="yyy"
export TF_VAR_AWS_DEFAULT_REGION="zzz"
```
- Update `contrib/terraform/aws/terraform.tfvars` with your data. By default, the Terraform scripts use Ubuntu 18.04 LTS (Bionic) as base image. If you want to change this behaviour, see note "Using other distrib than Ubuntu" below.
- Create an AWS EC2 SSH Key
- Run with `terraform apply --var-file="credentials.tfvars"` or `terraform apply` depending if you exported your AWS credentials
Example:
```commandline
terraform apply -var-file=credentials.tfvars
```
- Terraform automatically creates an Ansible Inventory file called `hosts` with the created infrastructure in the directory `inventory`
- Ansible will automatically generate an ssh config file for your bastion hosts. To connect to hosts with ssh using bastion host use generated `ssh-bastion.conf`. Ansible automatically detects bastion and changes `ssh_args`
```commandline
ssh -F ./ssh-bastion.conf user@$ip
```
- Once the infrastructure is created, you can run the kubespray playbooks and supply inventory/hosts with the `-i` flag.
Example (this one assumes you are using Ubuntu)
```commandline
ansible-playbook -i ./inventory/hosts ./cluster.yml -e ansible_user=ubuntu -b --become-user=root --flush-cache
```
## Using other distrib than Ubuntu***
To leverage a Linux distribution other than Ubuntu 18.04 (Bionic) LTS for your Terraform configurations, you can adjust the AMI search filters within the 'data "aws_ami" "distro"' block by utilizing variables in your `terraform.tfvars` file. This approach ensures a flexible configuration that adapts to various Linux distributions without directly modifying the core Terraform files.
### Example Usages
- **Debian Jessie**: To configure the usage of Debian Jessie, insert the subsequent lines into your `terraform.tfvars`:
```hcl
ami_name_pattern = "debian-jessie-amd64-hvm-*"
ami_owners = ["379101102735"]
```
- **Ubuntu 16.04**: To utilize Ubuntu 16.04 instead, apply the following configuration in your `terraform.tfvars`:
```hcl
ami_name_pattern = "ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-*"
ami_owners = ["099720109477"]
```
- **Centos 7**: For employing Centos 7, incorporate these lines into your `terraform.tfvars`:
```hcl
ami_name_pattern = "dcos-centos7-*"
ami_owners = ["688023202711"]
```
## Connecting to Kubernetes
You can use the following set of commands to get the kubeconfig file from your newly created cluster. Before running the commands, make sure you are in the project's root folder.
```commandline
# Get the controller's IP address.
CONTROLLER_HOST_NAME=$(cat ./inventory/hosts | grep "\[kube_control_plane\]" -A 1 | tail -n 1)
CONTROLLER_IP=$(cat ./inventory/hosts | grep $CONTROLLER_HOST_NAME | grep ansible_host | cut -d'=' -f2)
# Get the hostname of the load balancer.
LB_HOST=$(cat inventory/hosts | grep apiserver_loadbalancer_domain_name | cut -d'"' -f2)
# Get the controller's SSH fingerprint.
ssh-keygen -R $CONTROLLER_IP > /dev/null 2>&1
ssh-keyscan -H $CONTROLLER_IP >> ~/.ssh/known_hosts 2>/dev/null
# Get the kubeconfig from the controller.
mkdir -p ~/.kube
ssh -F ssh-bastion.conf centos@$CONTROLLER_IP "sudo chmod 644 /etc/kubernetes/admin.conf"
scp -F ssh-bastion.conf centos@$CONTROLLER_IP:/etc/kubernetes/admin.conf ~/.kube/config
sed -i "s^server:.*^server: https://$LB_HOST:6443^" ~/.kube/config
kubectl get nodes
```
## Troubleshooting
### Remaining AWS IAM Instance Profile
If the cluster was destroyed without using Terraform it is possible that
the AWS IAM Instance Profiles still remain. To delete them you can use
the `AWS CLI` with the following command:
```commandline
aws iam delete-instance-profile --region <region_name> --instance-profile-name <profile_name>
```
### Ansible Inventory doesn't get created
It could happen that Terraform doesn't create an Ansible Inventory file automatically. If this is the case copy the output after `inventory=` and create a file named `hosts`in the directory `inventory` and paste the inventory into the file.
## Architecture
Pictured is an AWS Infrastructure created with this Terraform project distributed over two Availability Zones.
================================================
FILE: contrib/terraform/aws/create-infrastructure.tf
================================================
terraform {
required_version = ">= 0.12.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.0"
}
}
}
provider "aws" {
access_key = var.AWS_ACCESS_KEY_ID
secret_key = var.AWS_SECRET_ACCESS_KEY
region = var.AWS_DEFAULT_REGION
}
data "aws_availability_zones" "available" {}
/*
* Calling modules who create the initial AWS VPC / AWS ELB
* and AWS IAM Roles for Kubernetes Deployment
*/
module "aws-vpc" {
source = "./modules/vpc"
aws_cluster_name = var.aws_cluster_name
aws_vpc_cidr_block = var.aws_vpc_cidr_block
aws_avail_zones = data.aws_availability_zones.available.names
aws_cidr_subnets_private = var.aws_cidr_subnets_private
aws_cidr_subnets_public = var.aws_cidr_subnets_public
default_tags = var.default_tags
}
module "aws-nlb" {
source = "./modules/nlb"
aws_cluster_name = var.aws_cluster_name
aws_vpc_id = module.aws-vpc.aws_vpc_id
aws_avail_zones = data.aws_availability_zones.available.names
aws_subnet_ids_public = module.aws-vpc.aws_subnet_ids_public
aws_nlb_api_port = var.aws_nlb_api_port
k8s_secure_api_port = var.k8s_secure_api_port
default_tags = var.default_tags
}
module "aws-iam" {
source = "./modules/iam"
aws_cluster_name = var.aws_cluster_name
}
/*
* Create Bastion Instances in AWS
*
*/
resource "aws_instance" "bastion-server" {
ami = data.aws_ami.distro.id
instance_type = var.aws_bastion_size
count = var.aws_bastion_num
associate_public_ip_address = true
subnet_id = element(module.aws-vpc.aws_subnet_ids_public, count.index)
vpc_security_group_ids = module.aws-vpc.aws_security_group
key_name = var.AWS_SSH_KEY_NAME
tags = merge(var.default_tags, tomap({
Name = "kubernetes-${var.aws_cluster_name}-bastion-${count.index}"
Cluster = var.aws_cluster_name
Role = "bastion-${var.aws_cluster_name}-${count.index}"
}))
}
/*
* Create K8s Master and worker nodes and etcd instances
*
*/
resource "aws_instance" "k8s-master" {
ami = data.aws_ami.distro.id
instance_type = var.aws_kube_master_size
count = var.aws_kube_master_num
subnet_id = element(module.aws-vpc.aws_subnet_ids_private, count.index)
vpc_security_group_ids = module.aws-vpc.aws_security_group
root_block_device {
volume_size = var.aws_kube_master_disk_size
}
iam_instance_profile = module.aws-iam.kube_control_plane-profile
key_name = var.AWS_SSH_KEY_NAME
tags = merge(var.default_tags, tomap({
Name = "kubernetes-${var.aws_cluster_name}-master${count.index}"
"kubernetes.io/cluster/${var.aws_cluster_name}" = "member"
Role = "master"
}))
}
resource "aws_lb_target_group_attachment" "tg-attach_master_nodes" {
count = var.aws_kube_master_num
target_group_arn = module.aws-nlb.aws_nlb_api_tg_arn
target_id = element(aws_instance.k8s-master.*.private_ip, count.index)
}
resource "aws_instance" "k8s-etcd" {
ami = data.aws_ami.distro.id
instance_type = var.aws_etcd_size
count = var.aws_etcd_num
subnet_id = element(module.aws-vpc.aws_subnet_ids_private, count.index)
vpc_security_group_ids = module.aws-vpc.aws_security_group
root_block_device {
volume_size = var.aws_etcd_disk_size
}
key_name = var.AWS_SSH_KEY_NAME
tags = merge(var.default_tags, tomap({
Name = "kubernetes-${var.aws_cluster_name}-etcd${count.index}"
"kubernetes.io/cluster/${var.aws_cluster_name}" = "member"
Role = "etcd"
}))
}
resource "aws_instance" "k8s-worker" {
ami = data.aws_ami.distro.id
instance_type = var.aws_kube_worker_size
count = var.aws_kube_worker_num
subnet_id = element(module.aws-vpc.aws_subnet_ids_private, count.index)
vpc_security_group_ids = module.aws-vpc.aws_security_group
root_block_device {
volume_size = var.aws_kube_worker_disk_size
}
iam_instance_profile = module.aws-iam.kube-worker-profile
key_name = var.AWS_SSH_KEY_NAME
tags = merge(var.default_tags, tomap({
Name = "kubernetes-${var.aws_cluster_name}-worker${count.index}"
"kubernetes.io/cluster/${var.aws_cluster_name}" = "member"
Role = "worker"
}))
}
/*
* Create Kubespray Inventory File
*
*/
data "template_file" "inventory" {
template = file("${path.module}/templates/inventory.tpl")
vars = {
public_ip_address_bastion = join("\n", formatlist("bastion ansible_host=%s", aws_instance.bastion-server.*.public_ip))
connection_strings_master = join("\n", formatlist("%s ansible_host=%s", aws_instance.k8s-master.*.private_dns, aws_instance.k8s-master.*.private_ip))
connection_strings_node = join("\n", formatlist("%s ansible_host=%s", aws_instance.k8s-worker.*.private_dns, aws_instance.k8s-worker.*.private_ip))
list_master = join("\n", aws_instance.k8s-master.*.private_dns)
list_node = join("\n", aws_instance.k8s-worker.*.private_dns)
connection_strings_etcd = join("\n", formatlist("%s ansible_host=%s", aws_instance.k8s-etcd.*.private_dns, aws_instance.k8s-etcd.*.private_ip))
list_etcd = join("\n", ((var.aws_etcd_num > 0) ? (aws_instance.k8s-etcd.*.private_dns) : (aws_instance.k8s-master.*.private_dns)))
nlb_api_fqdn = "apiserver_loadbalancer_domain_name=\"${module.aws-nlb.aws_nlb_api_fqdn}\""
}
}
resource "null_resource" "inventories" {
provisioner "local-exec" {
command = "echo '${data.template_file.inventory.rendered}' > ${var.inventory_file}"
}
triggers = {
template = data.template_file.inventory.rendered
}
}
================================================
FILE: contrib/terraform/aws/credentials.tfvars.example
================================================
#AWS Access Key
AWS_ACCESS_KEY_ID = ""
#AWS Secret Key
AWS_SECRET_ACCESS_KEY = ""
#EC2 SSH Key Name
AWS_SSH_KEY_NAME = ""
#AWS Region
AWS_DEFAULT_REGION = "eu-central-1"
================================================
FILE: contrib/terraform/aws/modules/iam/main.tf
================================================
#Add AWS Roles for Kubernetes
resource "aws_iam_role" "kube_control_plane" {
name = "kubernetes-${var.aws_cluster_name}-master"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Principal": {
"Service": "ec2.amazonaws.com"
}
}
]
}
EOF
}
resource "aws_iam_role" "kube-worker" {
name = "kubernetes-${var.aws_cluster_name}-node"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Principal": {
"Service": "ec2.amazonaws.com"
}
}
]
}
EOF
}
#Add AWS Policies for Kubernetes
resource "aws_iam_role_policy" "kube_control_plane" {
name = "kubernetes-${var.aws_cluster_name}-master"
role = aws_iam_role.kube_control_plane.id
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["ec2:*"],
"Resource": ["*"]
},
{
"Effect": "Allow",
"Action": ["elasticloadbalancing:*"],
"Resource": ["*"]
},
{
"Effect": "Allow",
"Action": ["route53:*"],
"Resource": ["*"]
},
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::kubernetes-*"
]
}
]
}
EOF
}
resource "aws_iam_role_policy" "kube-worker" {
name = "kubernetes-${var.aws_cluster_name}-node"
role = aws_iam_role.kube-worker.id
policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::kubernetes-*"
]
},
{
"Effect": "Allow",
"Action": "ec2:Describe*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "ec2:AttachVolume",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "ec2:DetachVolume",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": ["route53:*"],
"Resource": ["*"]
},
{
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:GetRepositoryPolicy",
"ecr:DescribeRepositories",
"ecr:ListImages",
"ecr:BatchGetImage"
],
"Resource": "*"
}
]
}
EOF
}
#Create AWS Instance Profiles
resource "aws_iam_instance_profile" "kube_control_plane" {
name = "kube_${var.aws_cluster_name}_master_profile"
role = aws_iam_role.kube_control_plane.name
}
resource "aws_iam_instance_profile" "kube-worker" {
name = "kube_${var.aws_cluster_name}_node_profile"
role = aws_iam_role.kube-worker.name
}
================================================
FILE: contrib/terraform/aws/modules/iam/outputs.tf
================================================
output "kube_control_plane-profile" {
value = aws_iam_instance_profile.kube_control_plane.name
}
output "kube-worker-profile" {
value = aws_iam_instance_profile.kube-worker.name
}
================================================
FILE: contrib/terraform/aws/modules/iam/variables.tf
================================================
variable "aws_cluster_name" {
description = "Name of Cluster"
}
================================================
FILE: contrib/terraform/aws/modules/nlb/main.tf
================================================
# Create a new AWS NLB for K8S API
resource "aws_lb" "aws-nlb-api" {
name = "kubernetes-nlb-${var.aws_cluster_name}"
load_balancer_type = "network"
subnets = length(var.aws_subnet_ids_public) <= length(var.aws_avail_zones) ? var.aws_subnet_ids_public : slice(var.aws_subnet_ids_public, 0, length(var.aws_avail_zones))
idle_timeout = 400
enable_cross_zone_load_balancing = true
tags = merge(var.default_tags, tomap({
Name = "kubernetes-${var.aws_cluster_name}-nlb-api"
}))
}
# Create a new AWS NLB Instance Target Group
resource "aws_lb_target_group" "aws-nlb-api-tg" {
name = "kubernetes-nlb-tg-${var.aws_cluster_name}"
port = var.k8s_secure_api_port
protocol = "TCP"
target_type = "ip"
vpc_id = var.aws_vpc_id
health_check {
healthy_threshold = 2
unhealthy_threshold = 2
interval = 30
protocol = "HTTPS"
path = "/healthz"
}
}
# Create a new AWS NLB Listener listen to target group
resource "aws_lb_listener" "aws-nlb-api-listener" {
load_balancer_arn = aws_lb.aws-nlb-api.arn
port = var.aws_nlb_api_port
protocol = "TCP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.aws-nlb-api-tg.arn
}
}
================================================
FILE: contrib/terraform/aws/modules/nlb/outputs.tf
================================================
output "aws_nlb_api_id" {
value = aws_lb.aws-nlb-api.id
}
output "aws_nlb_api_fqdn" {
value = aws_lb.aws-nlb-api.dns_name
}
output "aws_nlb_api_tg_arn" {
value = aws_lb_target_group.aws-nlb-api-tg.arn
}
================================================
FILE: contrib/terraform/aws/modules/nlb/variables.tf
================================================
variable "aws_cluster_name" {
description = "Name of Cluster"
}
variable "aws_vpc_id" {
description = "AWS VPC ID"
}
variable "aws_nlb_api_port" {
description = "Port for AWS NLB"
}
variable "k8s_secure_api_port" {
description = "Secure Port of K8S API Server"
}
variable "aws_avail_zones" {
description = "Availability Zones Used"
type = list(string)
}
variable "aws_subnet_ids_public" {
description = "IDs of Public Subnets"
type = list(string)
}
variable "default_tags" {
description = "Tags for all resources"
type = map(string)
}
================================================
FILE: contrib/terraform/aws/modules/vpc/main.tf
================================================
resource "aws_vpc" "cluster-vpc" {
cidr_block = var.aws_vpc_cidr_block
#DNS Related Entries
enable_dns_support = true
enable_dns_hostnames = true
tags = merge(var.default_tags, tomap({
Name = "kubernetes-${var.aws_cluster_name}-vpc"
}))
}
resource "aws_eip" "cluster-nat-eip" {
count = length(var.aws_cidr_subnets_public)
vpc = true
}
resource "aws_internet_gateway" "cluster-vpc-internetgw" {
vpc_id = aws_vpc.cluster-vpc.id
tags = merge(var.default_tags, tomap({
Name = "kubernetes-${var.aws_cluster_name}-internetgw"
}))
}
resource "aws_subnet" "cluster-vpc-subnets-public" {
vpc_id = aws_vpc.cluster-vpc.id
count = length(var.aws_cidr_subnets_public)
availability_zone = element(var.aws_avail_zones, count.index % length(var.aws_avail_zones))
cidr_block = element(var.aws_cidr_subnets_public, count.index)
tags = merge(var.default_tags, tomap({
Name = "kubernetes-${var.aws_cluster_name}-${element(var.aws_avail_zones, count.index)}-public"
"kubernetes.io/cluster/${var.aws_cluster_name}" = "shared"
"kubernetes.io/role/elb" = "1"
}))
}
resource "aws_nat_gateway" "cluster-nat-gateway" {
count = length(var.aws_cidr_subnets_public)
allocation_id = element(aws_eip.cluster-nat-eip.*.id, count.index)
subnet_id = element(aws_subnet.cluster-vpc-subnets-public.*.id, count.index)
}
resource "aws_subnet" "cluster-vpc-subnets-private" {
vpc_id = aws_vpc.cluster-vpc.id
count = length(var.aws_cidr_subnets_private)
availability_zone = element(var.aws_avail_zones, count.index % length(var.aws_avail_zones))
cidr_block = element(var.aws_cidr_subnets_private, count.index)
tags = merge(var.default_tags, tomap({
Name = "kubernetes-${var.aws_cluster_name}-${element(var.aws_avail_zones, count.index)}-private"
"kubernetes.io/cluster/${var.aws_cluster_name}" = "shared"
"kubernetes.io/role/internal-elb" = "1"
}))
}
#Routing in VPC
#TODO: Do we need two routing tables for each subnet for redundancy or is one enough?
resource "aws_route_table" "kubernetes-public" {
vpc_id = aws_vpc.cluster-vpc.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.cluster-vpc-internetgw.id
}
tags = merge(var.default_tags, tomap({
Name = "kubernetes-${var.aws_cluster_name}-routetable-public"
}))
}
resource "aws_route_table" "kubernetes-private" {
count = length(var.aws_cidr_subnets_private)
vpc_id = aws_vpc.cluster-vpc.id
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = element(aws_nat_gateway.cluster-nat-gateway.*.id, count.index)
}
tags = merge(var.default_tags, tomap({
Name = "kubernetes-${var.aws_cluster_name}-routetable-private-${count.index}"
}))
}
resource "aws_route_table_association" "kubernetes-public" {
count = length(var.aws_cidr_subnets_public)
subnet_id = element(aws_subnet.cluster-vpc-subnets-public.*.id, count.index)
route_table_id = aws_route_table.kubernetes-public.id
}
resource "aws_route_table_association" "kubernetes-private" {
count = length(var.aws_cidr_subnets_private)
subnet_id = element(aws_subnet.cluster-vpc-subnets-private.*.id, count.index)
route_table_id = element(aws_route_table.kubernetes-private.*.id, count.index)
}
#Kubernetes Security Groups
resource "aws_security_group" "kubernetes" {
name = "kubernetes-${var.aws_cluster_name}-securitygroup"
vpc_id = aws_vpc.cluster-vpc.id
tags = merge(var.default_tags, tomap({
Name = "kubernetes-${var.aws_cluster_name}-securitygroup"
}))
}
resource "aws_security_group_rule" "allow-all-ingress" {
type = "ingress"
from_port = 0
to_port = 65535
protocol = "-1"
cidr_blocks = [var.aws_vpc_cidr_block]
security_group_id = aws_security_group.kubernetes.id
}
resource "aws_security_group_rule" "allow-all-egress" {
type = "egress"
from_port = 0
to_port = 65535
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
security_group_id = aws_security_group.kubernetes.id
}
resource "aws_security_group_rule" "allow-ssh-connections" {
type = "ingress"
from_port = 22
to_port = 22
protocol = "TCP"
cidr_blocks = ["0.0.0.0/0"]
security_group_id = aws_security_group.kubernetes.id
}
================================================
FILE: contrib/terraform/aws/modules/vpc/outputs.tf
================================================
output "aws_vpc_id" {
value = aws_vpc.cluster-vpc.id
}
output "aws_subnet_ids_private" {
value = aws_subnet.cluster-vpc-subnets-private.*.id
}
output "aws_subnet_ids_public" {
value = aws_subnet.cluster-vpc-subnets-public.*.id
}
output "aws_security_group" {
value = aws_security_group.kubernetes.*.id
}
output "default_tags" {
value = var.default_tags
}
================================================
FILE: contrib/terraform/aws/modules/vpc/variables.tf
================================================
variable "aws_vpc_cidr_block" {
description = "CIDR Blocks for AWS VPC"
}
variable "aws_cluster_name" {
description = "Name of Cluster"
}
variable "aws_avail_zones" {
description = "AWS Availability Zones Used"
type = list(string)
}
variable "aws_cidr_subnets_private" {
description = "CIDR Blocks for private subnets in Availability zones"
type = list(string)
}
variable "aws_cidr_subnets_public" {
description = "CIDR Blocks for public subnets in Availability zones"
type = list(string)
}
variable "default_tags" {
description = "Default tags for all resources"
type = map(string)
}
================================================
FILE: contrib/terraform/aws/output.tf
================================================
output "bastion_ip" {
value = join("\n", aws_instance.bastion-server.*.public_ip)
}
output "masters" {
value = join("\n", aws_instance.k8s-master.*.private_ip)
}
output "workers" {
value = join("\n", aws_instance.k8s-worker.*.private_ip)
}
output "etcd" {
value = join("\n", ((var.aws_etcd_num > 0) ? (aws_instance.k8s-etcd.*.private_ip) : (aws_instance.k8s-master.*.private_ip)))
}
output "aws_nlb_api_fqdn" {
value = "${module.aws-nlb.aws_nlb_api_fqdn}:${var.aws_nlb_api_port}"
}
output "inventory" {
value = data.template_file.inventory.rendered
}
output "default_tags" {
value = var.default_tags
}
================================================
FILE: contrib/terraform/aws/sample-inventory/cluster.tfvars
================================================
#Global Vars
aws_cluster_name = "devtest"
#VPC Vars
aws_vpc_cidr_block = "10.250.192.0/18"
aws_cidr_subnets_private = ["10.250.192.0/20", "10.250.208.0/20"]
aws_cidr_subnets_public = ["10.250.224.0/20", "10.250.240.0/20"]
#Bastion Host
aws_bastion_num = 1
aws_bastion_size = "t2.medium"
#Kubernetes Cluster
aws_kube_master_num = 3
aws_kube_master_size = "t2.medium"
aws_kube_master_disk_size = 50
aws_etcd_num = 3
aws_etcd_size = "t2.medium"
aws_etcd_disk_size = 50
aws_kube_worker_num = 4
aws_kube_worker_size = "t2.medium"
aws_kube_worker_disk_size = 50
#Settings AWS NLB
aws_nlb_api_port = 6443
k8s_secure_api_port = 6443
default_tags = {
# Env = "devtest" # Product = "kubernetes"
}
inventory_file = "../../../inventory/hosts"
## Credentials
#AWS Access Key
AWS_ACCESS_KEY_ID = ""
#AWS Secret Key
AWS_SECRET_ACCESS_KEY = ""
#EC2 SSH Key Name
AWS_SSH_KEY_NAME = ""
#AWS Region
AWS_DEFAULT_REGION = "eu-central-1"
================================================
FILE: contrib/terraform/aws/templates/inventory.tpl
================================================
[all]
${connection_strings_master}
${connection_strings_node}
${connection_strings_etcd}
${public_ip_address_bastion}
[bastion]
${public_ip_address_bastion}
[kube_control_plane]
${list_master}
[kube_node]
${list_node}
[etcd]
${list_etcd}
[calico_rr]
[k8s_cluster:children]
kube_node
kube_control_plane
calico_rr
[k8s_cluster:vars]
${nlb_api_fqdn}
================================================
FILE: contrib/terraform/aws/terraform.tfvars
================================================
#Global Vars
aws_cluster_name = "devtest"
#VPC Vars
aws_vpc_cidr_block = "10.250.192.0/18"
aws_cidr_subnets_private = ["10.250.192.0/20", "10.250.208.0/20"]
aws_cidr_subnets_public = ["10.250.224.0/20", "10.250.240.0/20"]
# single AZ deployment
#aws_cidr_subnets_private = ["10.250.192.0/20"]
#aws_cidr_subnets_public = ["10.250.224.0/20"]
# 3+ AZ deployment
#aws_cidr_subnets_private = ["10.250.192.0/24","10.250.193.0/24","10.250.194.0/24","10.250.195.0/24"]
#aws_cidr_subnets_public = ["10.250.224.0/24","10.250.225.0/24","10.250.226.0/24","10.250.227.0/24"]
#Bastion Host
aws_bastion_num = 1
aws_bastion_size = "t3.small"
#Kubernetes Cluster
aws_kube_master_num = 3
aws_kube_master_size = "t3.medium"
aws_kube_master_disk_size = 50
aws_etcd_num = 0
aws_etcd_size = "t3.medium"
aws_etcd_disk_size = 50
aws_kube_worker_num = 4
aws_kube_worker_size = "t3.medium"
aws_kube_worker_disk_size = 50
#Settings AWS ELB
aws_nlb_api_port = 6443
k8s_secure_api_port = 6443
default_tags = {
# Env = "devtest"
# Product = "kubernetes"
}
inventory_file = "../../../inventory/hosts"
================================================
FILE: contrib/terraform/aws/terraform.tfvars.example
================================================
#Global Vars
aws_cluster_name = "devtest"
#VPC Vars
aws_vpc_cidr_block = "10.250.192.0/18"
aws_cidr_subnets_private = ["10.250.192.0/20","10.250.208.0/20"]
aws_cidr_subnets_public = ["10.250.224.0/20","10.250.240.0/20"]
aws_avail_zones = ["eu-central-1a","eu-central-1b"]
#Bastion Host
aws_bastion_num = 1
aws_bastion_size = "t3.small"
#Kubernetes Cluster
aws_kube_master_num = 3
aws_kube_master_size = "t3.medium"
aws_kube_master_disk_size = 50
aws_etcd_num = 3
aws_etcd_size = "t3.medium"
aws_etcd_disk_size = 50
aws_kube_worker_num = 4
aws_kube_worker_size = "t3.medium"
aws_kube_worker_disk_size = 50
#Settings AWS ELB
aws_nlb_api_port = 6443
k8s_secure_api_port = 6443
default_tags = { }
inventory_file = "../../../inventory/hosts"
================================================
FILE: contrib/terraform/aws/variables.tf
================================================
variable "AWS_ACCESS_KEY_ID" {
description = "AWS Access Key"
}
variable "AWS_SECRET_ACCESS_KEY" {
description = "AWS Secret Key"
}
variable "AWS_SSH_KEY_NAME" {
description = "Name of the SSH keypair to use in AWS."
}
variable "AWS_DEFAULT_REGION" {
description = "AWS Region"
}
//General Cluster Settings
variable "aws_cluster_name" {
description = "Name of AWS Cluster"
}
variable "ami_name_pattern" {
description = "The name pattern to use for AMI lookup"
type = string
default = "debian-10-amd64-*"
}
variable "ami_virtualization_type" {
description = "The virtualization type to use for AMI lookup"
type = string
default = "hvm"
}
variable "ami_owners" {
description = "The owners to use for AMI lookup"
type = list(string)
default = ["136693071363"]
}
data "aws_ami" "distro" {
most_recent = true
filter {
name = "name"
values = [var.ami_name_pattern]
}
filter {
name = "virtualization-type"
values = [var.ami_virtualization_type]
}
owners = var.ami_owners
}
//AWS VPC Variables
variable "aws_vpc_cidr_block" {
description = "CIDR Block for VPC"
}
variable "aws_cidr_subnets_private" {
description = "CIDR Blocks for private subnets in Availability Zones"
type = list(string)
}
variable "aws_cidr_subnets_public" {
description = "CIDR Blocks for public subnets in Availability Zones"
type = list(string)
}
//AWS EC2 Settings
variable "aws_bastion_size" {
description = "EC2 Instance Size of Bastion Host"
}
/*
* AWS EC2 Settings
* The number should be divisable by the number of used
* AWS Availability Zones without an remainder.
*/
variable "aws_bastion_num" {
description = "Number of Bastion Nodes"
}
variable "aws_kube_master_num" {
description = "Number of Kubernetes Master Nodes"
}
variable "aws_kube_master_disk_size" {
description = "Disk size for Kubernetes Master Nodes (in GiB)"
}
variable "aws_kube_master_size" {
description = "Instance size of Kube Master Nodes"
}
variable "aws_etcd_num" {
description = "Number of etcd Nodes"
}
variable "aws_etcd_disk_size" {
description = "Disk size for etcd Nodes (in GiB)"
}
variable "aws_etcd_size" {
description = "Instance size of etcd Nodes"
}
variable "aws_kube_worker_num" {
description = "Number of Kubernetes Worker Nodes"
}
variable "aws_kube_worker_disk_size" {
description = "Disk size for Kubernetes Worker Nodes (in GiB)"
}
variable "aws_kube_worker_size" {
description = "Instance size of Kubernetes Worker Nodes"
}
/*
* AWS NLB Settings
*
*/
variable "aws_nlb_api_port" {
description = "Port for AWS NLB"
}
variable "k8s_secure_api_port" {
description = "Secure Port of K8S API Server"
}
variable "default_tags" {
description = "Default tags for all resources"
type = map(string)
}
variable "inventory_file" {
description = "Where to store the generated inventory file"
}
================================================
FILE: contrib/terraform/exoscale/README.md
================================================
# Kubernetes on Exoscale with Terraform
Provision a Kubernetes cluster on [Exoscale](https://www.exoscale.com/) using Terraform and Kubespray
## Overview
The setup looks like following
```text
Kubernetes cluster
+-----------------------+
+---------------+ | +--------------+ |
| | | | +--------------+ |
| API server LB +---------> | | | |
| | | | | Master/etcd | |
+---------------+ | | | node(s) | |
| +-+ | |
| +--------------+ |
| ^ |
| | |
| v |
+---------------+ | +--------------+ |
| | | | +--------------+ |
| Ingress LB +---------> | | | |
| | | | | Worker | |
+---------------+ | | | node(s) | |
| +-+ | |
| +--------------+ |
+-----------------------+
```
## Requirements
* Terraform 0.13.0 or newer (0.12 also works if you modify the provider block to include version and remove all `versions.tf` files)
## Quickstart
NOTE: *Assumes you are at the root of the kubespray repo*
Copy the sample inventory for your cluster and copy the default terraform variables.
```bash
CLUSTER=my-exoscale-cluster
cp -r inventory/sample inventory/$CLUSTER
cp contrib/terraform/exoscale/default.tfvars inventory/$CLUSTER/
cd inventory/$CLUSTER
```
Edit `default.tfvars` to match your setup. You MUST, at the very least, change `ssh_public_keys`.
```bash
# Ensure $EDITOR points to your favorite editor, e.g., vim, emacs, VS Code, etc.
$EDITOR default.tfvars
```
For authentication you can use the credentials file `~/.cloudstack.ini` or `./cloudstack.ini`.
The file should look like something like this:
```ini
[cloudstack]
key = <API key>
secret = <API secret>
```
Follow the [Exoscale IAM Quick-start](https://community.exoscale.com/documentation/iam/quick-start/) to learn how to generate API keys.
### Encrypted credentials
To have the credentials encrypted at rest, you can use [sops](https://github.com/mozilla/sops) and only decrypt the credentials at runtime.
```bash
cat << EOF > cloudstack.ini
[cloudstack]
key =
secret =
EOF
sops --encrypt --in-place --pgp <PGP key fingerprint> cloudstack.ini
sops cloudstack.ini
```
Run terraform to create the infrastructure
```bash
terraform init ../../contrib/terraform/exoscale
terraform apply -var-file default.tfvars ../../contrib/terraform/exoscale
```
If your cloudstack credentials file is encrypted using sops, run the following:
```bash
terraform init ../../contrib/terraform/exoscale
sops exec-file -no-fifo cloudstack.ini 'CLOUDSTACK_CONFIG={} terraform apply -var-file default.tfvars ../../contrib/terraform/exoscale'
```
You should now have a inventory file named `inventory.ini` that you can use with kubespray.
You can now copy your inventory file and use it with kubespray to set up a cluster.
You can type `terraform output` to find out the IP addresses of the nodes, as well as control-plane and data-plane load-balancer.
It is a good idea to check that you have basic SSH connectivity to the nodes. You can do that by:
```bash
ansible -i inventory.ini -m ping all
```
Example to use this with the default sample inventory:
```bash
ansible-playbook -i inventory.ini ../../cluster.yml -b -v
```
## Teardown
The Kubernetes cluster cannot create any load-balancers or disks, hence, teardown is as simple as Terraform destroy:
```bash
terraform destroy -var-file default.tfvars ../../contrib/terraform/exoscale
```
## Variables
### Required
* `ssh_public_keys`: List of public SSH keys to install on all machines
* `zone`: The zone where to run the cluster
* `machines`: Machines to provision. Key of this object will be used as the name of the machine
* `node_type`: The role of this node *(master|worker)*
* `size`: The size to use
* `boot_disk`: The boot disk to use
* `image_name`: Name of the image
* `root_partition_size`: Size *(in GB)* for the root partition
* `ceph_partition_size`: Size *(in GB)* for the partition for rook to use as ceph storage. *(Set to 0 to disable)*
* `node_local_partition_size`: Size *(in GB)* for the partition for node-local-storage. *(Set to 0 to disable)*
* `ssh_whitelist`: List of IP ranges (CIDR) that will be allowed to ssh to the nodes
* `api_server_whitelist`: List of IP ranges (CIDR) that will be allowed to connect to the API server
* `nodeport_whitelist`: List of IP ranges (CIDR) that will be allowed to connect to the kubernetes nodes on port 30000-32767 (kubernetes nodeports)
### Optional
* `prefix`: Prefix to use for all resources, required to be unique for all clusters in the same project *(Defaults to `default`)*
An example variables file can be found `default.tfvars`
## Known limitations
### Only single disk
Since Exoscale doesn't support additional disks to be mounted onto an instance, this script has the ability to create partitions for [Rook](https://rook.io/) and [node-local-storage](https://kubernetes.io/docs/concepts/storage/volumes/#local).
### No Kubernetes API
The current solution doesn't use the [Exoscale Kubernetes cloud controller](https://github.com/exoscale/exoscale-cloud-controller-manager).
This means that we need to set up a HTTP(S) loadbalancer in front of all workers and set the Ingress controller to DaemonSet mode.
================================================
FILE: contrib/terraform/exoscale/default.tfvars
================================================
prefix = "default"
zone = "ch-gva-2"
inventory_file = "inventory.ini"
ssh_public_keys = [
# Put your public SSH key here
"ssh-rsa I-did-not-read-the-docs",
"ssh-rsa I-did-not-read-the-docs 2",
]
machines = {
"master-0" : {
"node_type" : "master",
"size" : "standard.medium",
"boot_disk" : {
"image_name" : "Linux Ubuntu 20.04 LTS 64-bit",
"root_partition_size" : 50,
"node_local_partition_size" : 0,
"ceph_partition_size" : 0
}
},
"worker-0" : {
"node_type" : "worker",
"size" : "standard.large",
"boot_disk" : {
"image_name" : "Linux Ubuntu 20.04 LTS 64-bit",
"root_partition_size" : 50,
"node_local_partition_size" : 0,
"ceph_partition_size" : 0
}
},
"worker-1" : {
"node_type" : "worker",
"size" : "standard.large",
"boot_disk" : {
"image_name" : "Linux Ubuntu 20.04 LTS 64-bit",
"root_partition_size" : 50,
"node_local_partition_size" : 0,
"ceph_partition_size" : 0
}
},
"worker-2" : {
"node_type" : "worker",
"size" : "standard.large",
"boot_disk" : {
"image_name" : "Linux Ubuntu 20.04 LTS 64-bit",
"root_partition_size" : 50,
"node_local_partition_size" : 0,
"ceph_partition_size" : 0
}
}
}
nodeport_whitelist = [
"0.0.0.0/0"
]
ssh_whitelist = [
"0.0.0.0/0"
]
api_server_whitelist = [
"0.0.0.0/0"
]
================================================
FILE: contrib/terraform/exoscale/main.tf
================================================
provider "exoscale" {}
module "kubernetes" {
source = "./modules/kubernetes-cluster"
prefix = var.prefix
zone = var.zone
machines = var.machines
ssh_public_keys = var.ssh_public_keys
ssh_whitelist = var.ssh_whitelist
api_server_whitelist = var.api_server_whitelist
nodeport_whitelist = var.nodeport_whitelist
}
#
# Generate ansible inventory
#
data "template_file" "inventory" {
template = file("${path.module}/templates/inventory.tpl")
vars = {
connection_strings_master = join("\n", formatlist("%s ansible_user=ubuntu ansible_host=%s ip=%s etcd_member_name=etcd%d",
keys(module.kubernetes.master_ip_addresses),
values(module.kubernetes.master_ip_addresses).*.public_ip,
values(module.kubernetes.master_ip_addresses).*.private_ip,
range(1, length(module.kubernetes.master_ip_addresses) + 1)))
connection_strings_worker = join("\n", formatlist("%s ansible_user=ubuntu ansible_host=%s ip=%s",
keys(module.kubernetes.worker_ip_addresses),
values(module.kubernetes.worker_ip_addresses).*.public_ip,
values(module.kubernetes.worker_ip_addresses).*.private_ip))
list_master = join("\n", keys(module.kubernetes.master_ip_addresses))
list_worker = join("\n", keys(module.kubernetes.worker_ip_addresses))
api_lb_ip_address = module.kubernetes.control_plane_lb_ip_address
}
}
resource "null_resource" "inventories" {
provisioner "local-exec" {
command = "echo '${data.template_file.inventory.rendered}' > ${var.inventory_file}"
}
triggers = {
template = data.template_file.inventory.rendered
}
}
================================================
FILE: contrib/terraform/exoscale/modules/kubernetes-cluster/main.tf
================================================
data "exoscale_template" "os_image" {
for_each = var.machines
zone = var.zone
name = each.value.boot_disk.image_name
}
data "exoscale_compute_instance" "master_nodes" {
for_each = exoscale_compute_instance.master
id = each.value.id
zone = var.zone
}
data "exoscale_compute_instance" "worker_nodes" {
for_each = exoscale_compute_instance.worker
id = each.value.id
zone = var.zone
}
resource "exoscale_private_network" "private_network" {
zone = var.zone
name = "${var.prefix}-network"
start_ip = cidrhost(var.private_network_cidr, 1)
# cidr -1 = Broadcast address
# cidr -2 = DHCP server address (exoscale specific)
end_ip = cidrhost(var.private_network_cidr, -3)
netmask = cidrnetmask(var.private_network_cidr)
}
resource "exoscale_compute_instance" "master" {
for_each = {
for name, machine in var.machines :
name => machine
if machine.node_type == "master"
}
name = "${var.prefix}-${each.key}"
template_id = data.exoscale_template.os_image[each.key].id
type = each.value.size
disk_size = each.value.boot_disk.root_partition_size + each.value.boot_disk.node_local_partition_size + each.value.boot_disk.ceph_partition_size
state = "Running"
zone = var.zone
security_group_ids = [exoscale_security_group.master_sg.id]
network_interface {
network_id = exoscale_private_network.private_network.id
}
elastic_ip_ids = [exoscale_elastic_ip.control_plane_lb.id]
user_data = templatefile(
"${path.module}/templates/cloud-init.tmpl",
{
eip_ip_address = exoscale_elastic_ip.ingress_controller_lb.ip_address
node_local_partition_size = each.value.boot_disk.node_local_partition_size
ceph_partition_size = each.value.boot_disk.ceph_partition_size
root_partition_size = each.value.boot_disk.root_partition_size
node_type = "master"
ssh_public_keys = var.ssh_public_keys
}
)
}
resource "exoscale_compute_instance" "worker" {
for_each = {
for name, machine in var.machines :
name => machine
if machine.node_type == "worker"
}
name = "${var.prefix}-${each.key}"
template_id = data.exoscale_template.os_image[each.key].id
type = each.value.size
disk_size = each.value.boot_disk.root_partition_size + each.value.boot_disk.node_local_partition_size + each.value.boot_disk.ceph_partition_size
state = "Running"
zone = var.zone
security_group_ids = [exoscale_security_group.worker_sg.id]
network_interface {
network_id = exoscale_private_network.private_network.id
}
elastic_ip_ids = [exoscale_elastic_ip.ingress_controller_lb.id]
user_data = templatefile(
"${path.module}/templates/cloud-init.tmpl",
{
eip_ip_address = exoscale_elastic_ip.ingress_controller_lb.ip_address
node_local_partition_size = each.value.boot_disk.node_local_partition_size
ceph_partition_size = each.value.boot_disk.ceph_partition_size
root_partition_size = each.value.boot_disk.root_partition_size
node_type = "worker"
ssh_public_keys = var.ssh_public_keys
}
)
}
resource "exoscale_security_group" "master_sg" {
name = "${var.prefix}-master-sg"
description = "Security group for Kubernetes masters"
}
resource "exoscale_security_group_rule" "master_sg_rule_ssh" {
security_group_id = exoscale_security_group.master_sg.id
for_each = toset(var.ssh_whitelist)
# SSH
type = "INGRESS"
start_port = 22
end_port = 22
protocol = "TCP"
cidr = each.value
}
resource "exoscale_security_group_rule" "master_sg_rule_k8s_api" {
security_group_id = exoscale_security_group.master_sg.id
for_each = toset(var.api_server_whitelist)
# Kubernetes API
type = "INGRESS"
start_port = 6443
end_port = 6443
protocol = "TCP"
cidr = each.value
}
resource "exoscale_security_group" "worker_sg" {
name = "${var.prefix}-worker-sg"
description = "security group for kubernetes worker nodes"
}
resource "exoscale_security_group_rule" "worker_sg_rule_ssh" {
security_group_id = exoscale_security_group.worker_sg.id
# SSH
for_each = toset(var.ssh_whitelist)
type = "INGRESS"
start_port = 22
end_port = 22
protocol = "TCP"
cidr = each.value
}
resource "exoscale_security_group_rule" "worker_sg_rule_http" {
security_group_id = exoscale_security_group.worker_sg.id
# HTTP(S)
for_each = toset(["80", "443"])
type = "INGRESS"
start_port = each.value
end_port = each.value
protocol = "TCP"
cidr = "0.0.0.0/0"
}
resource "exoscale_security_group_rule" "worker_sg_rule_nodeport" {
security_group_id = exoscale_security_group.worker_sg.id
# HTTP(S)
for_each = toset(var.nodeport_whitelist)
type = "INGRESS"
start_port = 30000
end_port = 32767
protocol = "TCP"
cidr = each.value
}
resource "exoscale_elastic_ip" "ingress_controller_lb" {
zone = var.zone
healthcheck {
mode = "http"
port = 80
uri = "/healthz"
interval = 10
timeout = 2
strikes_ok = 2
strikes_fail = 3
}
}
resource "exoscale_elastic_ip" "control_plane_lb" {
zone = var.zone
healthcheck {
mode = "tcp"
port = 6443
interval = 10
timeout = 2
strikes_ok = 2
strikes_fail = 3
}
}
================================================
FILE: contrib/terraform/exoscale/modules/kubernetes-cluster/output.tf
================================================
output "master_ip_addresses" {
value = {
for key, instance in exoscale_compute_instance.master :
instance.name => {
"private_ip" = contains(keys(data.exoscale_compute_instance.master_nodes), key) ? data.exoscale_compute_instance.master_nodes[key].private_network_ip_addresses[0] : ""
"public_ip" = exoscale_compute_instance.master[key].ip_address
}
}
}
output "worker_ip_addresses" {
value = {
for key, instance in exoscale_compute_instance.worker :
instance.name => {
"private_ip" = contains(keys(data.exoscale_compute_instance.worker_nodes), key) ? data.exoscale_compute_instance.worker_nodes[key].private_network_ip_addresses[0] : ""
"public_ip" = exoscale_compute_instance.worker[key].ip_address
}
}
}
output "cluster_private_network_cidr" {
value = var.private_network_cidr
}
output "ingress_controller_lb_ip_address" {
value = exoscale_elastic_ip.ingress_controller_lb.ip_address
}
output "control_plane_lb_ip_address" {
value = exoscale_elastic_ip.control_plane_lb.ip_address
}
================================================
FILE: contrib/terraform/exoscale/modules/kubernetes-cluster/templates/cloud-init.tmpl
================================================
#cloud-config
%{ if ceph_partition_size > 0 || node_local_partition_size > 0}
bootcmd:
- [ cloud-init-per, once, move-second-header, sgdisk, --move-second-header, /dev/vda ]
%{ if node_local_partition_size > 0 }
# Create partition for node local storage
- [ cloud-init-per, once, create-node-local-part, parted, --script, /dev/vda, 'mkpart extended ext4 ${root_partition_size}GB %{ if ceph_partition_size == 0 }-1%{ else }${root_partition_size + node_local_partition_size}GB%{ endif }' ]
- [ cloud-init-per, once, create-fs-node-local-part, mkfs.ext4, /dev/vda2 ]
%{ endif }
%{ if ceph_partition_size > 0 }
# Create partition for rook to use for ceph
- [ cloud-init-per, once, create-ceph-part, parted, --script, /dev/vda, 'mkpart extended ${root_partition_size + node_local_partition_size}GB -1' ]
%{ endif }
%{ endif }
ssh_authorized_keys:
%{ for ssh_public_key in ssh_public_keys ~}
- ${ssh_public_key}
%{ endfor ~}
write_files:
- path: /etc/netplan/eth1.yaml
content: |
network:
version: 2
ethernets:
eth1:
dhcp4: true
%{ if node_type == "worker" }
# TODO: When a VM is seen as healthy and is added to the EIP loadbalancer
# pool it no longer can send traffic back to itself via the EIP IP
# address.
# Remove this if it ever gets solved.
- path: /etc/netplan/20-eip-fix.yaml
content: |
network:
version: 2
ethernets:
"lo:0":
match:
name: lo
dhcp4: false
addresses:
- ${eip_ip_address}/32
%{ endif }
runcmd:
- netplan apply
%{ if node_local_partition_size > 0 }
- mkdir -p /mnt/disks/node-local-storage
- chown nobody:nogroup /mnt/disks/node-local-storage
- mount /dev/vda2 /mnt/disks/node-local-storage
%{ endif }
================================================
FILE: contrib/terraform/e
gitextract_cnwkyuho/ ├── .ansible-lint ├── .ansible-lint-ignore ├── .editorconfig ├── .gitattributes ├── .github/ │ ├── ISSUE_TEMPLATE/ │ │ ├── bug-report.yaml │ │ ├── config.yml │ │ ├── enhancement.yaml │ │ └── failing-test.yaml │ ├── PULL_REQUEST_TEMPLATE.md │ ├── dependabot.yml │ └── workflows/ │ ├── auto-label-os.yml │ ├── upgrade-patch-versions-schedule.yml │ └── upgrade-patch-versions.yml ├── .gitignore ├── .gitlab-ci/ │ ├── build.yml │ ├── kubevirt.yml │ ├── lint.yml │ ├── molecule.yml │ ├── terraform.yml │ └── vagrant.yml ├── .gitlab-ci.yml ├── .gitmodules ├── .md_style.rb ├── .mdlrc ├── .nojekyll ├── .pre-commit-config.yaml ├── .yamllint ├── CHANGELOG.md ├── CNAME ├── CONTRIBUTING.md ├── Dockerfile ├── LICENSE ├── OWNERS ├── OWNERS_ALIASES ├── README.md ├── RELEASE.md ├── SECURITY_CONTACTS ├── Vagrantfile ├── _config.yml ├── ansible.cfg ├── cluster.yml ├── code-of-conduct.md ├── contrib/ │ ├── aws_iam/ │ │ ├── kubernetes-master-policy.json │ │ ├── kubernetes-master-role.json │ │ ├── kubernetes-minion-policy.json │ │ └── kubernetes-minion-role.json │ ├── aws_inventory/ │ │ ├── kubespray-aws-inventory.py │ │ └── requirements.txt │ ├── azurerm/ │ │ ├── .gitignore │ │ ├── README.md │ │ ├── apply-rg.sh │ │ ├── clear-rg.sh │ │ ├── generate-inventory.sh │ │ ├── generate-inventory.yml │ │ ├── generate-inventory_2.yml │ │ ├── generate-templates.yml │ │ ├── group_vars/ │ │ │ └── all │ │ └── roles/ │ │ ├── generate-inventory/ │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ └── inventory.j2 │ │ ├── generate-inventory_2/ │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ ├── inventory.j2 │ │ │ └── loadbalancer_vars.j2 │ │ └── generate-templates/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ ├── tasks/ │ │ │ └── main.yml │ │ └── templates/ │ │ ├── availability-sets.json │ │ ├── bastion.json │ │ ├── clear-rg.json │ │ ├── masters.json │ │ ├── minions.json │ │ ├── network.json │ │ └── storage.json │ ├── collection.sh │ ├── offline/ │ │ ├── README.md │ │ ├── docker-daemon.json │ │ ├── generate_list.sh │ │ ├── generate_list.yml │ │ ├── manage-offline-container-images.sh │ │ ├── manage-offline-files.sh │ │ ├── nginx.conf │ │ ├── registries.conf │ │ └── upload2artifactory.py │ ├── os-services/ │ │ ├── os-services.yml │ │ └── roles/ │ │ └── prepare/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ └── tasks/ │ │ └── main.yml │ └── terraform/ │ ├── aws/ │ │ ├── .gitignore │ │ ├── README.md │ │ ├── create-infrastructure.tf │ │ ├── credentials.tfvars.example │ │ ├── modules/ │ │ │ ├── iam/ │ │ │ │ ├── main.tf │ │ │ │ ├── outputs.tf │ │ │ │ └── variables.tf │ │ │ ├── nlb/ │ │ │ │ ├── main.tf │ │ │ │ ├── outputs.tf │ │ │ │ └── variables.tf │ │ │ └── vpc/ │ │ │ ├── main.tf │ │ │ ├── outputs.tf │ │ │ └── variables.tf │ │ ├── output.tf │ │ ├── sample-inventory/ │ │ │ └── cluster.tfvars │ │ ├── templates/ │ │ │ └── inventory.tpl │ │ ├── terraform.tfvars │ │ ├── terraform.tfvars.example │ │ └── variables.tf │ ├── exoscale/ │ │ ├── README.md │ │ ├── default.tfvars │ │ ├── main.tf │ │ ├── modules/ │ │ │ └── kubernetes-cluster/ │ │ │ ├── main.tf │ │ │ ├── output.tf │ │ │ ├── templates/ │ │ │ │ └── cloud-init.tmpl │ │ │ ├── variables.tf │ │ │ └── versions.tf │ │ ├── output.tf │ │ ├── sample-inventory/ │ │ │ └── cluster.tfvars │ │ ├── templates/ │ │ │ └── inventory.tpl │ │ ├── variables.tf │ │ └── versions.tf │ ├── gcp/ │ │ ├── README.md │ │ ├── generate-inventory.sh │ │ ├── main.tf │ │ ├── modules/ │ │ │ └── kubernetes-cluster/ │ │ │ ├── main.tf │ │ │ ├── output.tf │ │ │ └── variables.tf │ │ ├── output.tf │ │ ├── tfvars.json │ │ └── variables.tf │ ├── hetzner/ │ │ ├── README.md │ │ ├── default.tfvars │ │ ├── main.tf │ │ ├── modules/ │ │ │ ├── kubernetes-cluster/ │ │ │ │ ├── main.tf │ │ │ │ ├── output.tf │ │ │ │ ├── templates/ │ │ │ │ │ └── cloud-init.tmpl │ │ │ │ ├── variables.tf │ │ │ │ └── versions.tf │ │ │ └── kubernetes-cluster-flatcar/ │ │ │ ├── main.tf │ │ │ ├── outputs.tf │ │ │ ├── templates/ │ │ │ │ └── machine.yaml.tmpl │ │ │ ├── variables.tf │ │ │ └── versions.tf │ │ ├── output.tf │ │ ├── sample-inventory/ │ │ │ └── cluster.tfvars │ │ ├── templates/ │ │ │ └── inventory.tpl │ │ ├── variables.tf │ │ └── versions.tf │ ├── openstack/ │ │ ├── .gitignore │ │ ├── README.md │ │ ├── kubespray.tf │ │ ├── modules/ │ │ │ ├── compute/ │ │ │ │ ├── ansible_bastion_template.txt │ │ │ │ ├── main.tf │ │ │ │ ├── outputs.tf │ │ │ │ ├── templates/ │ │ │ │ │ └── cloudinit.yaml.tmpl │ │ │ │ ├── variables.tf │ │ │ │ └── versions.tf │ │ │ ├── ips/ │ │ │ │ ├── main.tf │ │ │ │ ├── outputs.tf │ │ │ │ ├── variables.tf │ │ │ │ └── versions.tf │ │ │ ├── loadbalancer/ │ │ │ │ ├── main.tf │ │ │ │ ├── variables.tf │ │ │ │ └── versions.tf │ │ │ └── network/ │ │ │ ├── main.tf │ │ │ ├── outputs.tf │ │ │ ├── variables.tf │ │ │ └── versions.tf │ │ ├── variables.tf │ │ └── versions.tf │ ├── terraform.py │ ├── upcloud/ │ │ ├── README.md │ │ ├── cluster-settings.tfvars │ │ ├── main.tf │ │ ├── modules/ │ │ │ └── kubernetes-cluster/ │ │ │ ├── main.tf │ │ │ ├── output.tf │ │ │ ├── variables.tf │ │ │ └── versions.tf │ │ ├── output.tf │ │ ├── sample-inventory/ │ │ │ └── cluster.tfvars │ │ ├── templates/ │ │ │ └── inventory.tpl │ │ ├── variables.tf │ │ └── versions.tf │ └── vsphere/ │ ├── README.md │ ├── default.tfvars │ ├── main.tf │ ├── modules/ │ │ └── kubernetes-cluster/ │ │ ├── main.tf │ │ ├── output.tf │ │ ├── templates/ │ │ │ ├── cloud-init.tpl │ │ │ ├── metadata.tpl │ │ │ └── vapp-cloud-init.tpl │ │ ├── variables.tf │ │ └── versions.tf │ ├── output.tf │ ├── sample-inventory/ │ │ └── cluster.tfvars │ ├── templates/ │ │ └── inventory.tpl │ ├── variables.tf │ └── versions.tf ├── docs/ │ ├── CNI/ │ │ ├── calico.md │ │ ├── cilium.md │ │ ├── cni.md │ │ ├── flannel.md │ │ ├── kube-ovn.md │ │ ├── kube-router.md │ │ ├── macvlan.md │ │ └── multus.md │ ├── CRI/ │ │ ├── containerd.md │ │ ├── cri-o.md │ │ ├── docker.md │ │ ├── gvisor.md │ │ └── kata-containers.md │ ├── CSI/ │ │ ├── aws-ebs-csi.md │ │ ├── azure-csi.md │ │ ├── cinder-csi.md │ │ ├── gcp-pd-csi.md │ │ └── vsphere-csi.md │ ├── _sidebar.md │ ├── advanced/ │ │ ├── arch.md │ │ ├── cert_manager.md │ │ ├── dns-stack.md │ │ ├── downloads.md │ │ ├── gcp-lb.md │ │ ├── kubernetes-reliability.md │ │ ├── netcheck.md │ │ ├── ntp.md │ │ ├── proxy.md │ │ └── registry.md │ ├── ansible/ │ │ ├── ansible.md │ │ ├── ansible_collection.md │ │ ├── inventory.md │ │ └── vars.md │ ├── calico_peer_example/ │ │ ├── new-york.yml │ │ └── paris.yml │ ├── cloud_controllers/ │ │ ├── openstack.md │ │ └── vsphere.md │ ├── cloud_providers/ │ │ ├── aws.md │ │ ├── azure.md │ │ └── cloud.md │ ├── developers/ │ │ ├── ci-setup.md │ │ ├── ci.md │ │ ├── test_cases.md │ │ └── vagrant.md │ ├── external_storage_provisioners/ │ │ ├── local_volume_provisioner.md │ │ └── scheduler_plugins.md │ ├── getting_started/ │ │ ├── comparisons.md │ │ ├── getting-started.md │ │ └── setting-up-your-first-cluster.md │ ├── ingress/ │ │ ├── alb_ingress_controller.md │ │ ├── kube-vip.md │ │ └── metallb.md │ ├── operating_systems/ │ │ ├── amazonlinux.md │ │ ├── bootstrap-os.md │ │ ├── fcos.md │ │ ├── flatcar.md │ │ ├── kylinlinux.md │ │ ├── openeuler.md │ │ ├── opensuse.md │ │ ├── rhel.md │ │ └── uoslinux.md │ ├── operations/ │ │ ├── cgroups.md │ │ ├── encrypting-secret-data-at-rest.md │ │ ├── etcd.md │ │ ├── ha-mode.md │ │ ├── hardening.md │ │ ├── integration.md │ │ ├── kernel-requirements.md │ │ ├── large-deployments.md │ │ ├── mirror.md │ │ ├── nodes.md │ │ ├── offline-environment.md │ │ ├── port-requirements.md │ │ ├── recover-control-plane.md │ │ └── upgrades.md │ ├── roadmap/ │ │ └── roadmap.md │ └── upgrades/ │ └── migrate_docker2containerd.md ├── extra_playbooks/ │ ├── files/ │ │ └── get_cinder_pvs.sh │ ├── migrate_openstack_provider.yml │ ├── upgrade-only-k8s.yml │ └── wait-for-cloud-init.yml ├── galaxy.yml ├── index.html ├── inventory/ │ ├── local/ │ │ └── hosts.ini │ └── sample/ │ ├── group_vars/ │ │ ├── all/ │ │ │ ├── all.yml │ │ │ ├── aws.yml │ │ │ ├── azure.yml │ │ │ ├── containerd.yml │ │ │ ├── coreos.yml │ │ │ ├── cri-o.yml │ │ │ ├── docker.yml │ │ │ ├── etcd.yml │ │ │ ├── gcp.yml │ │ │ ├── hcloud.yml │ │ │ ├── huaweicloud.yml │ │ │ ├── oci.yml │ │ │ ├── offline.yml │ │ │ ├── openstack.yml │ │ │ ├── upcloud.yml │ │ │ └── vsphere.yml │ │ └── k8s_cluster/ │ │ ├── addons.yml │ │ ├── k8s-cluster.yml │ │ ├── k8s-net-calico.yml │ │ ├── k8s-net-cilium.yml │ │ ├── k8s-net-custom-cni.yml │ │ ├── k8s-net-flannel.yml │ │ ├── k8s-net-kube-ovn.yml │ │ ├── k8s-net-kube-router.yml │ │ ├── k8s-net-macvlan.yml │ │ └── kube_control_plane.yml │ └── inventory.ini ├── logo/ │ ├── LICENSE │ └── usage_guidelines.md ├── meta/ │ └── runtime.yml ├── pipeline.Dockerfile ├── playbooks/ │ ├── ansible_version.yml │ ├── boilerplate.yml │ ├── cluster.yml │ ├── facts.yml │ ├── install_etcd.yml │ ├── internal_facts.yml │ ├── recover_control_plane.yml │ ├── remove_node.yml │ ├── reset.yml │ ├── scale.yml │ └── upgrade_cluster.yml ├── plugins/ │ └── modules/ │ └── kube.py ├── recover-control-plane.yml ├── remove-node.yml ├── remove_node.yml ├── requirements.txt ├── reset.yml ├── roles/ │ ├── adduser/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ ├── molecule/ │ │ │ └── default/ │ │ │ ├── converge.yml │ │ │ └── molecule.yml │ │ ├── tasks/ │ │ │ └── main.yml │ │ └── vars/ │ │ ├── coreos.yml │ │ ├── debian.yml │ │ └── redhat.yml │ ├── bastion-ssh-config/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ ├── molecule/ │ │ │ └── default/ │ │ │ ├── converge.yml │ │ │ └── molecule.yml │ │ ├── tasks/ │ │ │ └── main.yml │ │ └── templates/ │ │ └── ssh-bastion.conf.j2 │ ├── bootstrap-os/ │ │ └── tasks/ │ │ └── main.yml │ ├── bootstrap_os/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ ├── files/ │ │ │ └── bootstrap.sh │ │ ├── handlers/ │ │ │ └── main.yml │ │ ├── meta/ │ │ │ └── main.yml │ │ ├── molecule/ │ │ │ └── default/ │ │ │ ├── converge.yml │ │ │ ├── molecule.yml │ │ │ └── tests/ │ │ │ └── test_default.py │ │ ├── tasks/ │ │ │ ├── almalinux.yml │ │ │ ├── amzn.yml │ │ │ ├── centos.yml │ │ │ ├── debian.yml │ │ │ ├── fedora-coreos.yml │ │ │ ├── fedora.yml │ │ │ ├── flatcar.yml │ │ │ ├── main.yml │ │ │ ├── openEuler.yml │ │ │ ├── opensuse-leap.yml │ │ │ ├── opensuse-tumbleweed.yml │ │ │ ├── opensuse.yml │ │ │ ├── rhel.yml │ │ │ ├── rocky.yml │ │ │ └── ubuntu.yml │ │ └── vars/ │ │ ├── fedora-coreos.yml │ │ └── flatcar.yml │ ├── container-engine/ │ │ ├── containerd/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── handlers/ │ │ │ │ ├── main.yml │ │ │ │ └── reset.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── molecule/ │ │ │ │ └── default/ │ │ │ │ ├── converge.yml │ │ │ │ ├── molecule.yml │ │ │ │ └── verify.yml │ │ │ ├── tasks/ │ │ │ │ ├── main.yml │ │ │ │ └── reset.yml │ │ │ └── templates/ │ │ │ ├── config-v1.toml.j2 │ │ │ ├── config.toml.j2 │ │ │ ├── containerd.service.j2 │ │ │ ├── hosts.toml.j2 │ │ │ └── http-proxy.conf.j2 │ │ ├── containerd-common/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── vars/ │ │ │ ├── amazon.yml │ │ │ └── suse.yml │ │ ├── cri-dockerd/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── handlers/ │ │ │ │ └── main.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── molecule/ │ │ │ │ └── default/ │ │ │ │ ├── converge.yml │ │ │ │ ├── files/ │ │ │ │ │ ├── 10-mynet.conf │ │ │ │ │ ├── container.json │ │ │ │ │ └── sandbox.json │ │ │ │ ├── molecule.yml │ │ │ │ └── verify.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ ├── cri-dockerd.service.j2 │ │ │ └── cri-dockerd.socket.j2 │ │ ├── cri-o/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── handlers/ │ │ │ │ └── main.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── molecule/ │ │ │ │ └── default/ │ │ │ │ ├── converge.yml │ │ │ │ ├── molecule.yml │ │ │ │ └── verify.yml │ │ │ ├── tasks/ │ │ │ │ ├── load_vars.yml │ │ │ │ ├── main.yaml │ │ │ │ ├── reset.yml │ │ │ │ └── setup-amazon.yaml │ │ │ ├── templates/ │ │ │ │ ├── config.json.j2 │ │ │ │ ├── crio.conf.j2 │ │ │ │ ├── http-proxy.conf.j2 │ │ │ │ ├── mounts.conf.j2 │ │ │ │ ├── registry.conf.j2 │ │ │ │ └── unqualified.conf.j2 │ │ │ └── vars/ │ │ │ ├── v1.29.yml │ │ │ └── v1.31.yml │ │ ├── crictl/ │ │ │ ├── handlers/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ └── crictl.yaml.j2 │ │ ├── crun/ │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── docker/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── files/ │ │ │ │ └── cleanup-docker-orphans.sh │ │ │ ├── handlers/ │ │ │ │ └── main.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ ├── docker_plugin.yml │ │ │ │ ├── main.yml │ │ │ │ ├── pre-upgrade.yml │ │ │ │ ├── reset.yml │ │ │ │ ├── set_facts_dns.yml │ │ │ │ └── systemd.yml │ │ │ ├── templates/ │ │ │ │ ├── docker-dns.conf.j2 │ │ │ │ ├── docker-options.conf.j2 │ │ │ │ ├── docker-orphan-cleanup.conf.j2 │ │ │ │ ├── docker.service.j2 │ │ │ │ ├── fedora_docker.repo.j2 │ │ │ │ ├── http-proxy.conf.j2 │ │ │ │ └── rh_docker.repo.j2 │ │ │ └── vars/ │ │ │ ├── amazon.yml │ │ │ ├── clearlinux.yml │ │ │ ├── debian.yml │ │ │ ├── fedora.yml │ │ │ ├── kylin.yml │ │ │ ├── redhat.yml │ │ │ ├── suse.yml │ │ │ ├── ubuntu.yml │ │ │ └── uniontech.yml │ │ ├── gvisor/ │ │ │ ├── molecule/ │ │ │ │ └── default/ │ │ │ │ ├── converge.yml │ │ │ │ ├── files/ │ │ │ │ │ ├── 10-mynet.conf │ │ │ │ │ ├── container.json │ │ │ │ │ └── sandbox.json │ │ │ │ ├── molecule.yml │ │ │ │ └── verify.yml │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── kata-containers/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── molecule/ │ │ │ │ └── default/ │ │ │ │ ├── converge.yml │ │ │ │ ├── files/ │ │ │ │ │ ├── 10-mynet.conf │ │ │ │ │ ├── container.json │ │ │ │ │ └── sandbox.json │ │ │ │ ├── molecule.yml │ │ │ │ └── verify.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ ├── configuration-qemu.toml.j2 │ │ │ └── containerd-shim-kata-v2.j2 │ │ ├── molecule/ │ │ │ ├── files/ │ │ │ │ └── 10-mynet.conf │ │ │ ├── prepare.yml │ │ │ ├── templates/ │ │ │ │ ├── container.json.j2 │ │ │ │ └── sandbox.json.j2 │ │ │ ├── test_cri.yml │ │ │ └── test_runtime.yml │ │ ├── nerdctl/ │ │ │ ├── handlers/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ └── nerdctl.toml.j2 │ │ ├── runc/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── skopeo/ │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── tasks/ │ │ │ └── main.yml │ │ ├── validate-container-engine/ │ │ │ └── tasks/ │ │ │ └── main.yml │ │ └── youki/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ ├── molecule/ │ │ │ └── default/ │ │ │ ├── converge.yml │ │ │ ├── files/ │ │ │ │ ├── 10-mynet.conf │ │ │ │ ├── container.json │ │ │ │ └── sandbox.json │ │ │ ├── molecule.yml │ │ │ └── verify.yml │ │ └── tasks/ │ │ └── main.yml │ ├── download/ │ │ ├── meta/ │ │ │ └── main.yml │ │ ├── tasks/ │ │ │ ├── check_pull_required.yml │ │ │ ├── download_container.yml │ │ │ ├── download_file.yml │ │ │ ├── extract_file.yml │ │ │ ├── main.yml │ │ │ ├── prep_download.yml │ │ │ ├── prep_kubeadm_images.yml │ │ │ └── set_container_facts.yml │ │ └── templates/ │ │ └── kubeadm-images.yaml.j2 │ ├── dynamic_groups/ │ │ └── tasks/ │ │ └── main.yml │ ├── etcd/ │ │ ├── handlers/ │ │ │ ├── backup.yml │ │ │ ├── backup_cleanup.yml │ │ │ └── main.yml │ │ ├── meta/ │ │ │ └── main.yml │ │ ├── tasks/ │ │ │ ├── check_certs.yml │ │ │ ├── clean_v2_store.yml │ │ │ ├── configure.yml │ │ │ ├── gen_certs_script.yml │ │ │ ├── gen_nodes_certs_script.yml │ │ │ ├── install_docker.yml │ │ │ ├── install_host.yml │ │ │ ├── join_etcd-events_member.yml │ │ │ ├── join_etcd_member.yml │ │ │ ├── main.yml │ │ │ ├── refresh_config.yml │ │ │ └── upd_ca_trust.yml │ │ └── templates/ │ │ ├── etcd-docker.service.j2 │ │ ├── etcd-events-docker.service.j2 │ │ ├── etcd-events-host.service.j2 │ │ ├── etcd-events.env.j2 │ │ ├── etcd-events.j2 │ │ ├── etcd-host.service.j2 │ │ ├── etcd.env.j2 │ │ ├── etcd.j2 │ │ ├── make-ssl-etcd.sh.j2 │ │ └── openssl.conf.j2 │ ├── etcd_defaults/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ └── vars/ │ │ └── main.yml │ ├── etcdctl_etcdutl/ │ │ ├── tasks/ │ │ │ └── main.yml │ │ └── templates/ │ │ └── etcdctl.sh.j2 │ ├── helm-apps/ │ │ ├── README.md │ │ ├── meta/ │ │ │ ├── argument_specs.yml │ │ │ └── main.yml │ │ ├── tasks/ │ │ │ └── main.yml │ │ └── vars/ │ │ └── main.yml │ ├── kubernetes/ │ │ ├── client/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── control-plane/ │ │ │ ├── defaults/ │ │ │ │ └── main/ │ │ │ │ ├── etcd.yml │ │ │ │ ├── kube-proxy.yml │ │ │ │ ├── kube-scheduler.yml │ │ │ │ └── main.yml │ │ │ ├── handlers/ │ │ │ │ └── main.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ ├── check-api.yml │ │ │ │ ├── encrypt-at-rest.yml │ │ │ │ ├── kubeadm-backup.yml │ │ │ │ ├── kubeadm-etcd.yml │ │ │ │ ├── kubeadm-secondary.yml │ │ │ │ ├── kubeadm-setup.yml │ │ │ │ ├── kubeadm-upgrade.yml │ │ │ │ ├── main.yml │ │ │ │ └── pre-upgrade.yml │ │ │ ├── templates/ │ │ │ │ ├── admission-controls.yaml.j2 │ │ │ │ ├── apiserver-audit-policy.yaml.j2 │ │ │ │ ├── apiserver-audit-webhook-config.yaml.j2 │ │ │ │ ├── apiserver-tracing.yaml.j2 │ │ │ │ ├── eventratelimit.yaml.j2 │ │ │ │ ├── k8s-certs-renew.service.j2 │ │ │ │ ├── k8s-certs-renew.sh.j2 │ │ │ │ ├── k8s-certs-renew.timer.j2 │ │ │ │ ├── kubeadm-config.v1beta4.yaml.j2 │ │ │ │ ├── kubeadm-controlplane.yaml.j2 │ │ │ │ ├── kubescheduler-config.yaml.j2 │ │ │ │ ├── podnodeselector.yaml.j2 │ │ │ │ ├── podsecurity.yaml.j2 │ │ │ │ ├── resourcequota.yaml.j2 │ │ │ │ ├── secrets_encryption.yaml.j2 │ │ │ │ ├── webhook-authorization-config.yaml.j2 │ │ │ │ └── webhook-token-auth-config.yaml.j2 │ │ │ └── vars/ │ │ │ └── main.yaml │ │ ├── kubeadm/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── handlers/ │ │ │ │ └── main.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ ├── kubeadm_etcd_node.yml │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ └── kubeadm-client.conf.j2 │ │ ├── kubeadm_common/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── node/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── handlers/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ ├── facts.yml │ │ │ │ ├── install.yml │ │ │ │ ├── kubelet.yml │ │ │ │ ├── loadbalancer/ │ │ │ │ │ ├── haproxy.yml │ │ │ │ │ ├── kube-vip.yml │ │ │ │ │ └── nginx-proxy.yml │ │ │ │ └── main.yml │ │ │ ├── templates/ │ │ │ │ ├── http-proxy.conf.j2 │ │ │ │ ├── kubelet-config.v1beta1.yaml.j2 │ │ │ │ ├── kubelet.env.v1beta1.j2 │ │ │ │ ├── kubelet.service.j2 │ │ │ │ ├── loadbalancer/ │ │ │ │ │ ├── haproxy.cfg.j2 │ │ │ │ │ └── nginx.conf.j2 │ │ │ │ └── manifests/ │ │ │ │ ├── haproxy.manifest.j2 │ │ │ │ ├── kube-vip.manifest.j2 │ │ │ │ └── nginx-proxy.manifest.j2 │ │ │ └── vars/ │ │ │ ├── fedora.yml │ │ │ ├── ubuntu-18.yml │ │ │ ├── ubuntu-20.yml │ │ │ ├── ubuntu-22.yml │ │ │ └── ubuntu-24.yml │ │ ├── node-label/ │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── node-taint/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ └── tasks/ │ │ │ └── main.yml │ │ └── preinstall/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ ├── files/ │ │ │ └── dhclient_nodnsupdate │ │ ├── handlers/ │ │ │ └── main.yml │ │ ├── meta/ │ │ │ └── main.yml │ │ ├── tasks/ │ │ │ ├── 0010-swapoff.yml │ │ │ ├── 0020-set_facts.yml │ │ │ ├── 0040-verify-settings.yml │ │ │ ├── 0050-create_directories.yml │ │ │ ├── 0060-resolvconf.yml │ │ │ ├── 0061-systemd-resolved.yml │ │ │ ├── 0062-networkmanager-unmanaged-devices.yml │ │ │ ├── 0063-networkmanager-dns.yml │ │ │ ├── 0080-system-configurations.yml │ │ │ ├── 0081-ntp-configurations.yml │ │ │ ├── 0100-dhclient-hooks.yml │ │ │ ├── 0110-dhclient-hooks-undo.yml │ │ │ └── main.yml │ │ ├── templates/ │ │ │ ├── ansible_git.j2 │ │ │ ├── chrony.conf.j2 │ │ │ ├── dhclient_dnsupdate.sh.j2 │ │ │ ├── dhclient_dnsupdate_rh.sh.j2 │ │ │ ├── ntp.conf.j2 │ │ │ ├── resolvconf.j2 │ │ │ └── resolved.conf.j2 │ │ └── vars/ │ │ └── main.yml │ ├── kubernetes-apps/ │ │ ├── ansible/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ ├── templates/ │ │ │ │ ├── coredns-clusterrole.yml.j2 │ │ │ │ ├── coredns-clusterrolebinding.yml.j2 │ │ │ │ ├── coredns-config.yml.j2 │ │ │ │ ├── coredns-deployment.yml.j2 │ │ │ │ ├── coredns-poddisruptionbudget.yml.j2 │ │ │ │ ├── coredns-sa.yml.j2 │ │ │ │ ├── coredns-svc.yml.j2 │ │ │ │ ├── dns-autoscaler-clusterrole.yml.j2 │ │ │ │ ├── dns-autoscaler-clusterrolebinding.yml.j2 │ │ │ │ ├── dns-autoscaler-sa.yml.j2 │ │ │ │ ├── dns-autoscaler.yml.j2 │ │ │ │ ├── etcd_metrics-endpoints.yml.j2 │ │ │ │ ├── etcd_metrics-service.yml.j2 │ │ │ │ ├── nodelocaldns-config.yml.j2 │ │ │ │ ├── nodelocaldns-daemonset.yml.j2 │ │ │ │ ├── nodelocaldns-sa.yml.j2 │ │ │ │ └── nodelocaldns-second-daemonset.yml.j2 │ │ │ └── vars/ │ │ │ └── main.yml │ │ ├── argocd/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ └── argocd-namespace.yml.j2 │ │ ├── cluster_roles/ │ │ │ ├── files/ │ │ │ │ └── k8s-cluster-critical-pc.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ ├── namespace.j2 │ │ │ ├── node-crb.yml.j2 │ │ │ └── vsphere-rbac.yml.j2 │ │ ├── common_crds/ │ │ │ ├── gateway_api/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ └── tasks/ │ │ │ │ └── main.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ └── prometheus_operator_crds/ │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── container_engine_accelerator/ │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ └── nvidia_gpu/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ ├── templates/ │ │ │ │ ├── k8s-device-plugin-nvidia-daemonset.yml.j2 │ │ │ │ └── nvidia-driver-install-daemonset.yml.j2 │ │ │ └── vars/ │ │ │ ├── ubuntu-16.yml │ │ │ └── ubuntu-18.yml │ │ ├── container_runtimes/ │ │ │ ├── crun/ │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yaml │ │ │ │ └── templates/ │ │ │ │ └── runtimeclass-crun.yml │ │ │ ├── gvisor/ │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yaml │ │ │ │ └── templates/ │ │ │ │ └── runtimeclass-gvisor.yml.j2 │ │ │ ├── kata_containers/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yaml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yaml │ │ │ │ └── templates/ │ │ │ │ └── runtimeclass-kata-qemu.yml.j2 │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ └── youki/ │ │ │ ├── tasks/ │ │ │ │ └── main.yaml │ │ │ └── templates/ │ │ │ └── runtimeclass-youki.yml │ │ ├── csi_driver/ │ │ │ ├── aws_ebs/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── aws-ebs-csi-controllerservice-rbac.yml.j2 │ │ │ │ ├── aws-ebs-csi-controllerservice.yml.j2 │ │ │ │ ├── aws-ebs-csi-driver.yml.j2 │ │ │ │ └── aws-ebs-csi-nodeservice.yml.j2 │ │ │ ├── azuredisk/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ ├── azure-credential-check.yml │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── azure-csi-azuredisk-controller-rbac.yml.j2 │ │ │ │ ├── azure-csi-azuredisk-controller.yml.j2 │ │ │ │ ├── azure-csi-azuredisk-driver.yml.j2 │ │ │ │ ├── azure-csi-azuredisk-node-rbac.yml.j2 │ │ │ │ ├── azure-csi-azuredisk-node.yml.j2 │ │ │ │ ├── azure-csi-cloud-config-secret.yml.j2 │ │ │ │ └── azure-csi-cloud-config.j2 │ │ │ ├── cinder/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ ├── cinder-credential-check.yml │ │ │ │ │ ├── cinder-write-cacert.yml │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── cinder-csi-cloud-config-secret.yml.j2 │ │ │ │ ├── cinder-csi-cloud-config.j2 │ │ │ │ ├── cinder-csi-controllerplugin-rbac.yml.j2 │ │ │ │ ├── cinder-csi-controllerplugin.yml.j2 │ │ │ │ ├── cinder-csi-driver.yml.j2 │ │ │ │ ├── cinder-csi-nodeplugin-rbac.yml.j2 │ │ │ │ ├── cinder-csi-nodeplugin.yml.j2 │ │ │ │ └── cinder-csi-poddisruptionbudget.yml.j2 │ │ │ ├── csi_crd/ │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── volumegroupsnapshotclasses.yml.j2 │ │ │ │ ├── volumegroupsnapshotcontents.yml.j2 │ │ │ │ ├── volumegroupsnapshots.yml.j2 │ │ │ │ ├── volumesnapshotclasses.yml.j2 │ │ │ │ ├── volumesnapshotcontents.yml.j2 │ │ │ │ └── volumesnapshots.yml.j2 │ │ │ ├── gcp_pd/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── gcp-pd-csi-controller.yml.j2 │ │ │ │ ├── gcp-pd-csi-cred-secret.yml.j2 │ │ │ │ ├── gcp-pd-csi-node.yml.j2 │ │ │ │ ├── gcp-pd-csi-sc-regional.yml.j2 │ │ │ │ ├── gcp-pd-csi-sc-zonal.yml.j2 │ │ │ │ └── gcp-pd-csi-setup.yml.j2 │ │ │ ├── upcloud/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── upcloud-csi-controller.yml.j2 │ │ │ │ ├── upcloud-csi-cred-secret.yml.j2 │ │ │ │ ├── upcloud-csi-driver.yml.j2 │ │ │ │ ├── upcloud-csi-node.yml.j2 │ │ │ │ └── upcloud-csi-setup.yml.j2 │ │ │ └── vsphere/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ ├── main.yml │ │ │ │ └── vsphere-credentials-check.yml │ │ │ └── templates/ │ │ │ ├── vsphere-csi-cloud-config.j2 │ │ │ ├── vsphere-csi-controller-config.yml.j2 │ │ │ ├── vsphere-csi-controller-deployment.yml.j2 │ │ │ ├── vsphere-csi-controller-rbac.yml.j2 │ │ │ ├── vsphere-csi-controller-service.yml.j2 │ │ │ ├── vsphere-csi-driver.yml.j2 │ │ │ ├── vsphere-csi-namespace.yml.j2 │ │ │ ├── vsphere-csi-node-rbac.yml.j2 │ │ │ └── vsphere-csi-node.yml.j2 │ │ ├── external_cloud_controller/ │ │ │ ├── hcloud/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── external-hcloud-cloud-controller-manager-ds-with-networks.yml.j2 │ │ │ │ ├── external-hcloud-cloud-controller-manager-ds.yml.j2 │ │ │ │ ├── external-hcloud-cloud-role-bindings.yml.j2 │ │ │ │ ├── external-hcloud-cloud-secret.yml.j2 │ │ │ │ └── external-hcloud-cloud-service-account.yml.j2 │ │ │ ├── huaweicloud/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ ├── huaweicloud-credential-check.yml │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── external-huawei-cloud-config-secret.yml.j2 │ │ │ │ ├── external-huawei-cloud-config.j2 │ │ │ │ ├── external-huawei-cloud-controller-manager-ds.yml.j2 │ │ │ │ ├── external-huawei-cloud-controller-manager-role-bindings.yml.j2 │ │ │ │ └── external-huawei-cloud-controller-manager-roles.yml.j2 │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── oci/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── external-oci-cloud-config-secret.yml.j2 │ │ │ │ ├── external-oci-cloud-config.yml.j2 │ │ │ │ ├── external-oci-cloud-controller-manager-rbac.yml.j2 │ │ │ │ └── external-oci-cloud-controller-manager.yml.j2 │ │ │ ├── openstack/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ ├── main.yml │ │ │ │ │ └── openstack-credential-check.yml │ │ │ │ └── templates/ │ │ │ │ ├── external-openstack-cloud-config-secret.yml.j2 │ │ │ │ ├── external-openstack-cloud-config.j2 │ │ │ │ ├── external-openstack-cloud-controller-manager-ds.yml.j2 │ │ │ │ ├── external-openstack-cloud-controller-manager-role-bindings.yml.j2 │ │ │ │ └── external-openstack-cloud-controller-manager-roles.yml.j2 │ │ │ └── vsphere/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ ├── main.yml │ │ │ │ └── vsphere-credentials-check.yml │ │ │ └── templates/ │ │ │ ├── external-vsphere-cloud-controller-manager-ds.yml.j2 │ │ │ ├── external-vsphere-cloud-controller-manager-role-bindings.yml.j2 │ │ │ ├── external-vsphere-cloud-controller-manager-roles.yml.j2 │ │ │ ├── external-vsphere-cpi-cloud-config-secret.yml.j2 │ │ │ └── external-vsphere-cpi-cloud-config.j2 │ │ ├── external_provisioner/ │ │ │ ├── local_path_provisioner/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── local-path-storage-clusterrolebinding.yml.j2 │ │ │ │ ├── local-path-storage-cm.yml.j2 │ │ │ │ ├── local-path-storage-cr.yml.j2 │ │ │ │ ├── local-path-storage-deployment.yml.j2 │ │ │ │ ├── local-path-storage-ns.yml.j2 │ │ │ │ ├── local-path-storage-sa.yml.j2 │ │ │ │ └── local-path-storage-sc.yml.j2 │ │ │ ├── local_volume_provisioner/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ ├── basedirs.yml │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── local-volume-provisioner-clusterrole.yml.j2 │ │ │ │ ├── local-volume-provisioner-clusterrolebinding.yml.j2 │ │ │ │ ├── local-volume-provisioner-cm.yml.j2 │ │ │ │ ├── local-volume-provisioner-ds.yml.j2 │ │ │ │ ├── local-volume-provisioner-ns.yml.j2 │ │ │ │ ├── local-volume-provisioner-sa.yml.j2 │ │ │ │ └── local-volume-provisioner-sc.yml.j2 │ │ │ └── meta/ │ │ │ └── main.yml │ │ ├── helm/ │ │ │ ├── .gitkeep │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ ├── main.yml │ │ │ │ └── pyyaml-flatcar.yml │ │ │ └── vars/ │ │ │ ├── amazon.yml │ │ │ ├── centos-7.yml │ │ │ ├── centos.yml │ │ │ ├── debian.yml │ │ │ ├── fedora.yml │ │ │ ├── redhat-7.yml │ │ │ ├── redhat.yml │ │ │ ├── suse.yml │ │ │ └── ubuntu.yml │ │ ├── ingress_controller/ │ │ │ ├── alb_ingress_controller/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── alb-ingress-clusterrole.yml.j2 │ │ │ │ ├── alb-ingress-clusterrolebinding.yml.j2 │ │ │ │ ├── alb-ingress-deploy.yml.j2 │ │ │ │ ├── alb-ingress-ns.yml.j2 │ │ │ │ └── alb-ingress-sa.yml.j2 │ │ │ ├── cert_manager/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── cert-manager.crds.yml.j2 │ │ │ │ └── cert-manager.yml.j2 │ │ │ └── meta/ │ │ │ └── main.yml │ │ ├── kubelet-csr-approver/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ └── meta/ │ │ │ └── main.yml │ │ ├── meta/ │ │ │ └── main.yml │ │ ├── metallb/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ ├── layer2.yaml.j2 │ │ │ ├── layer3.yaml.j2 │ │ │ ├── metallb.yaml.j2 │ │ │ └── pools.yaml.j2 │ │ ├── metrics_server/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ ├── auth-delegator.yaml.j2 │ │ │ ├── auth-reader.yaml.j2 │ │ │ ├── metrics-apiservice.yaml.j2 │ │ │ ├── metrics-server-deployment.yaml.j2 │ │ │ ├── metrics-server-sa.yaml.j2 │ │ │ ├── metrics-server-service.yaml.j2 │ │ │ ├── resource-reader-clusterrolebinding.yaml.j2 │ │ │ └── resource-reader.yaml.j2 │ │ ├── node_feature_discovery/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ ├── nfd-api-crds.yaml.j2 │ │ │ ├── nfd-clusterrole.yaml.j2 │ │ │ ├── nfd-clusterrolebinding.yaml.j2 │ │ │ ├── nfd-gc.yaml.j2 │ │ │ ├── nfd-master-conf.yaml.j2 │ │ │ ├── nfd-master.yaml.j2 │ │ │ ├── nfd-ns.yaml.j2 │ │ │ ├── nfd-role.yaml.j2 │ │ │ ├── nfd-rolebinding.yaml.j2 │ │ │ ├── nfd-service.yaml.j2 │ │ │ ├── nfd-serviceaccount.yaml.j2 │ │ │ ├── nfd-topologyupdater-conf.yaml.j2 │ │ │ ├── nfd-worker-conf.yaml.j2 │ │ │ └── nfd-worker.yaml.j2 │ │ ├── persistent_volumes/ │ │ │ ├── aws-ebs-csi/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ └── aws-ebs-csi-storage-class.yml.j2 │ │ │ ├── azuredisk-csi/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ └── azure-csi-storage-class.yml.j2 │ │ │ ├── cinder-csi/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ └── cinder-csi-storage-class.yml.j2 │ │ │ ├── gcp-pd-csi/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ └── gcp-pd-csi-storage-class.yml.j2 │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ └── upcloud-csi/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ └── upcloud-csi-storage-class.yml.j2 │ │ ├── policy_controller/ │ │ │ ├── calico/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ ├── calico-kube-controllers.yml.j2 │ │ │ │ ├── calico-kube-cr.yml.j2 │ │ │ │ ├── calico-kube-crb.yml.j2 │ │ │ │ └── calico-kube-sa.yml.j2 │ │ │ └── meta/ │ │ │ └── main.yml │ │ ├── registry/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ ├── registry-cm.yml.j2 │ │ │ ├── registry-ing.yml.j2 │ │ │ ├── registry-ns.yml.j2 │ │ │ ├── registry-pvc.yml.j2 │ │ │ ├── registry-rs.yml.j2 │ │ │ ├── registry-sa.yml.j2 │ │ │ ├── registry-secrets.yml.j2 │ │ │ └── registry-svc.yml.j2 │ │ ├── scheduler_plugins/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ ├── appgroup.diktyo.x-k8s.io_appgroups.yaml.j2 │ │ │ ├── cm-scheduler-plugins.yaml.j2 │ │ │ ├── deploy-scheduler-plugins.yaml.j2 │ │ │ ├── namespace.yaml.j2 │ │ │ ├── networktopology.diktyo.x-k8s.io_networktopologies.yaml.j2 │ │ │ ├── rbac-scheduler-plugins.yaml.j2 │ │ │ ├── sa-scheduler-plugins.yaml.j2 │ │ │ ├── scheduling.x-k8s.io_elasticquotas.yaml.j2 │ │ │ ├── scheduling.x-k8s.io_podgroups.yaml.j2 │ │ │ └── topology.node.k8s.io_noderesourcetopologies.yaml.j2 │ │ ├── snapshots/ │ │ │ ├── cinder-csi/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ ├── tasks/ │ │ │ │ │ └── main.yml │ │ │ │ └── templates/ │ │ │ │ └── cinder-csi-snapshot-class.yml.j2 │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ └── snapshot-controller/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ ├── rbac-snapshot-controller.yml.j2 │ │ │ ├── snapshot-controller.yml.j2 │ │ │ └── snapshot-ns.yml.j2 │ │ └── utils/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ └── vars/ │ │ └── main.yml │ ├── kubespray-defaults/ │ │ └── tasks/ │ │ └── main.yml │ ├── kubespray_defaults/ │ │ ├── defaults/ │ │ │ └── main/ │ │ │ ├── download.yml │ │ │ └── main.yml │ │ └── vars/ │ │ └── main/ │ │ ├── checksums.yml │ │ └── main.yml │ ├── network_facts/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ ├── meta/ │ │ │ └── main.yml │ │ └── tasks/ │ │ └── main.yaml │ ├── network_plugin/ │ │ ├── calico/ │ │ │ ├── files/ │ │ │ │ └── openssl.conf │ │ │ ├── handlers/ │ │ │ │ └── main.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── rr/ │ │ │ │ ├── defaults/ │ │ │ │ │ └── main.yml │ │ │ │ └── tasks/ │ │ │ │ ├── main.yml │ │ │ │ ├── pre.yml │ │ │ │ └── update-node.yml │ │ │ ├── tasks/ │ │ │ │ ├── calico_apiserver_certs.yml │ │ │ │ ├── check.yml │ │ │ │ ├── install.yml │ │ │ │ ├── main.yml │ │ │ │ ├── peer_with_calico_rr.yml │ │ │ │ ├── peer_with_router.yml │ │ │ │ ├── pre.yml │ │ │ │ ├── repos.yml │ │ │ │ ├── reset.yml │ │ │ │ └── typha_certs.yml │ │ │ ├── templates/ │ │ │ │ ├── calico-apiserver-ns.yml.j2 │ │ │ │ ├── calico-apiserver.yml.j2 │ │ │ │ ├── calico-config.yml.j2 │ │ │ │ ├── calico-cr.yml.j2 │ │ │ │ ├── calico-crb.yml.j2 │ │ │ │ ├── calico-ipamconfig.yml.j2 │ │ │ │ ├── calico-node-sa.yml.j2 │ │ │ │ ├── calico-node.yml.j2 │ │ │ │ ├── calico-typha.yml.j2 │ │ │ │ ├── calicoctl.etcd.sh.j2 │ │ │ │ ├── calicoctl.kdd.sh.j2 │ │ │ │ ├── kubernetes-services-endpoint.yml.j2 │ │ │ │ └── make-ssl-calico.sh.j2 │ │ │ └── vars/ │ │ │ ├── amazon.yml │ │ │ ├── centos-9.yml │ │ │ ├── debian.yml │ │ │ ├── fedora.yml │ │ │ ├── opensuse.yml │ │ │ ├── redhat-9.yml │ │ │ ├── redhat.yml │ │ │ └── rocky-9.yml │ │ ├── calico_defaults/ │ │ │ └── defaults/ │ │ │ └── main.yml │ │ ├── cilium/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ ├── apply.yml │ │ │ │ ├── check.yml │ │ │ │ ├── install.yml │ │ │ │ ├── main.yml │ │ │ │ ├── reset.yml │ │ │ │ └── reset_iface.yml │ │ │ └── templates/ │ │ │ ├── cilium/ │ │ │ │ ├── cilium-bgp-advertisement.yml.j2 │ │ │ │ ├── cilium-bgp-cluster-config.yml.j2 │ │ │ │ ├── cilium-bgp-node-config-override.yml.j2 │ │ │ │ ├── cilium-bgp-peer-config.yml.j2 │ │ │ │ ├── cilium-bgp-peering-policy.yml.j2 │ │ │ │ └── cilium-loadbalancer-ip-pool.yml.j2 │ │ │ └── values.yaml.j2 │ │ ├── cni/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── custom_cni/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── flannel/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ ├── main.yml │ │ │ │ └── reset.yml │ │ │ └── templates/ │ │ │ ├── cni-flannel-rbac.yml.j2 │ │ │ └── cni-flannel.yml.j2 │ │ ├── kube-ovn/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ ├── cni-kube-ovn-crd.yml.j2 │ │ │ ├── cni-kube-ovn.yml.j2 │ │ │ └── cni-ovn.yml.j2 │ │ ├── kube-router/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── handlers/ │ │ │ │ └── main.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ ├── annotate.yml │ │ │ │ ├── main.yml │ │ │ │ └── reset.yml │ │ │ └── templates/ │ │ │ ├── cni-conf.json.j2 │ │ │ ├── kube-router.yml.j2 │ │ │ └── kubeconfig.yml.j2 │ │ ├── macvlan/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── files/ │ │ │ │ ├── ifdown-local │ │ │ │ ├── ifdown-macvlan │ │ │ │ ├── ifup-local │ │ │ │ └── ifup-macvlan │ │ │ ├── handlers/ │ │ │ │ └── main.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ ├── 10-macvlan.conf.j2 │ │ │ ├── 99-loopback.conf.j2 │ │ │ ├── centos-network-macvlan.cfg.j2 │ │ │ ├── centos-postdown-macvlan.cfg.j2 │ │ │ ├── centos-postup-macvlan.cfg.j2 │ │ │ ├── centos-routes-macvlan.cfg.j2 │ │ │ ├── coreos-device-macvlan.cfg.j2 │ │ │ ├── coreos-interface-macvlan.cfg.j2 │ │ │ ├── coreos-network-macvlan.cfg.j2 │ │ │ ├── coreos-service-nat_ouside.j2 │ │ │ └── debian-network-macvlan.cfg.j2 │ │ ├── multus/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ ├── files/ │ │ │ │ ├── multus-clusterrole.yml │ │ │ │ ├── multus-clusterrolebinding.yml │ │ │ │ ├── multus-crd.yml │ │ │ │ └── multus-serviceaccount.yml │ │ │ ├── meta/ │ │ │ │ └── main.yml │ │ │ ├── tasks/ │ │ │ │ └── main.yml │ │ │ └── templates/ │ │ │ └── multus-daemonset.yml.j2 │ │ ├── ovn4nfv/ │ │ │ └── tasks/ │ │ │ └── main.yml │ │ └── tasks/ │ │ └── main.yml │ ├── recover_control_plane/ │ │ ├── control-plane/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── etcd/ │ │ │ └── tasks/ │ │ │ ├── main.yml │ │ │ └── recover_lost_quorum.yml │ │ └── post-recover/ │ │ └── tasks/ │ │ └── main.yml │ ├── remove-node/ │ │ ├── post-remove/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── pre-remove/ │ │ │ └── tasks/ │ │ │ └── main.yml │ │ └── remove-etcd-node/ │ │ └── tasks/ │ │ └── main.yml │ ├── remove_node/ │ │ └── pre_remove/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ └── tasks/ │ │ └── main.yml │ ├── reset/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ └── tasks/ │ │ └── main.yml │ ├── system_packages/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ ├── tasks/ │ │ │ └── main.yml │ │ └── vars/ │ │ └── main.yml │ ├── upgrade/ │ │ ├── post-upgrade/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ └── tasks/ │ │ │ └── main.yml │ │ ├── pre-upgrade/ │ │ │ ├── defaults/ │ │ │ │ └── main.yml │ │ │ └── tasks/ │ │ │ └── main.yml │ │ └── system-upgrade/ │ │ └── tasks/ │ │ ├── apt.yml │ │ ├── main.yml │ │ └── yum.yml │ ├── validate_inventory/ │ │ ├── meta/ │ │ │ └── main.yml │ │ └── tasks/ │ │ └── main.yml │ └── win_nodes/ │ └── kubernetes_patch/ │ ├── defaults/ │ │ └── main.yml │ └── tasks/ │ └── main.yml ├── scale.yml ├── scripts/ │ ├── Dockerfile.j2 │ ├── assert-sorted-checksums.yml │ ├── collect-info.yaml │ ├── component_hash_update/ │ │ ├── pyproject.toml │ │ └── src/ │ │ └── component_hash_update/ │ │ ├── __init__.py │ │ ├── components.py │ │ ├── download.py │ │ └── list_releases.graphql │ ├── galaxy_version.py │ ├── gen_docs_sidebar.sh │ ├── get_node_ids.sh │ ├── gitlab-runner.sh │ ├── openstack-cleanup/ │ │ ├── .gitignore │ │ ├── README.md │ │ ├── main.py │ │ └── requirements.txt │ ├── pipeline.Dockerfile.j2 │ ├── propagate_ansible_variables.yml │ └── readme_versions.md.j2 ├── test-infra/ │ ├── image-builder/ │ │ ├── Makefile │ │ ├── OWNERS │ │ ├── README.md │ │ ├── cluster.yml │ │ ├── hosts.ini │ │ └── roles/ │ │ └── kubevirt-images/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ ├── tasks/ │ │ │ └── main.yml │ │ └── templates/ │ │ └── Dockerfile │ └── vagrant-docker/ │ ├── Dockerfile │ ├── README.md │ └── build.sh ├── tests/ │ ├── Makefile │ ├── ansible.cfg │ ├── cloud_playbooks/ │ │ ├── create-kubevirt.yml │ │ └── roles/ │ │ └── packet-ci/ │ │ ├── defaults/ │ │ │ └── main.yml │ │ ├── tasks/ │ │ │ └── main.yml │ │ ├── templates/ │ │ │ └── vm.yml.j2 │ │ └── vars/ │ │ └── main.yml │ ├── common_vars.yml │ ├── files/ │ │ ├── almalinux9-calico-ha-ebpf.yml │ │ ├── almalinux9-calico-nodelocaldns-secondary.yml │ │ ├── almalinux9-calico-remove-node │ │ ├── almalinux9-calico-remove-node.yml │ │ ├── almalinux9-calico.yml │ │ ├── almalinux9-crio.yml │ │ ├── almalinux9-docker.yml │ │ ├── almalinux9-kube-ovn.yml │ │ ├── amazon-linux-2-all-in-one.yml │ │ ├── custom_cni/ │ │ │ ├── README.md │ │ │ ├── cilium.yaml │ │ │ └── values.yaml │ │ ├── debian11-calico-collection.yml │ │ ├── debian11-calico-upgrade │ │ ├── debian11-calico-upgrade-once │ │ ├── debian11-calico-upgrade-once.yml │ │ ├── debian11-calico-upgrade.yml │ │ ├── debian11-custom-cni.yml │ │ ├── debian11-docker.yml │ │ ├── debian11-kubelet-csr-approver.yml │ │ ├── debian11-macvlan.yml │ │ ├── debian12-calico.yml │ │ ├── debian12-cilium-svc-proxy.yml │ │ ├── debian12-cilium.yml │ │ ├── debian12-custom-cni-helm.yml │ │ ├── debian12-docker.yml │ │ ├── debian13-calico.yml │ │ ├── debian13-cilium.yml │ │ ├── fedora39-calico-selinux.yml │ │ ├── fedora39-calico-swap-selinux.yml │ │ ├── fedora39-crio.yml │ │ ├── fedora39-kube-router.yml │ │ ├── fedora40-docker-calico.yml │ │ ├── fedora40-docker.calico │ │ ├── fedora40-flannel-crio-collection-scale.yml │ │ ├── fedora41-calico-selinux.yml │ │ ├── fedora41-calico-swap-selinux.yml │ │ ├── fedora41-crio.yml │ │ ├── fedora41-kube-router.yml │ │ ├── fedora42-calico.yml │ │ ├── flatcar4081-calico.yml │ │ ├── openeuler24-calico.yml │ │ ├── rockylinux10-calico.yml │ │ ├── rockylinux10-cilium │ │ ├── rockylinux10-cilium.yml │ │ ├── rockylinux9-calico.yml │ │ ├── rockylinux9-cilium │ │ ├── rockylinux9-cilium.yml │ │ ├── tf-elastx_ubuntu24-calico.yml │ │ ├── ubuntu22-all-in-one-docker.yml │ │ ├── ubuntu22-calico-all-in-one-upgrade │ │ ├── ubuntu22-calico-all-in-one-upgrade.yml │ │ ├── ubuntu22-calico-all-in-one.yml │ │ ├── ubuntu22-crio.yml │ │ ├── ubuntu24-all-in-one-docker.yml │ │ ├── ubuntu24-calico-all-in-one │ │ ├── ubuntu24-calico-all-in-one-hardening.yml │ │ ├── ubuntu24-calico-all-in-one.yml │ │ ├── ubuntu24-calico-dual-stack.rb │ │ ├── ubuntu24-calico-dual-stack.yml │ │ ├── ubuntu24-calico-etcd-datastore.yml │ │ ├── ubuntu24-calico-etcd-kubeadm-upgrade-ha │ │ ├── ubuntu24-calico-etcd-kubeadm-upgrade-ha.yml │ │ ├── ubuntu24-calico-etcd-kubeadm.yml │ │ ├── ubuntu24-calico-ha-recover │ │ ├── ubuntu24-calico-ha-recover-noquorum │ │ ├── ubuntu24-calico-ha-recover-noquorum.yml │ │ ├── ubuntu24-calico-ha-recover.yml │ │ ├── ubuntu24-calico-ha-wireguard.yml │ │ ├── ubuntu24-calico-ipv6only-stack.rb │ │ ├── ubuntu24-calico-ipv6only-stack.yml │ │ ├── ubuntu24-cilium-sep.yml │ │ ├── ubuntu24-crio-scale.yml │ │ ├── ubuntu24-crio-upgrade │ │ ├── ubuntu24-crio-upgrade.yml │ │ ├── ubuntu24-flannel-collection.yml │ │ ├── ubuntu24-flannel-ha-once.yml │ │ ├── ubuntu24-flannel-ha.yml │ │ ├── ubuntu24-flannel.yml │ │ ├── ubuntu24-ha-separate-etcd │ │ ├── ubuntu24-ha-separate-etcd.yml │ │ ├── ubuntu24-kube-router-sep.yml │ │ └── ubuntu24-kube-router-svc-proxy.yml │ ├── requirements.txt │ ├── scripts/ │ │ ├── check-templates.py │ │ ├── collection-build-install.sh │ │ ├── md-table/ │ │ │ ├── main.py │ │ │ └── table.md.j2 │ │ ├── molecule_run.sh │ │ ├── opentofu_install.sh │ │ ├── rebase.sh │ │ ├── testcases_run.sh │ │ ├── vagrant-install.sh │ │ ├── vagrant-validate.sh │ │ └── vagrant_clean.sh │ └── testcases/ │ ├── 000_install-hydrophone.yml │ ├── 010_check-apiserver.yml │ ├── 015_check-nodes-ready.yml │ ├── 020_check-pods-running.yml │ ├── 025_check-csr-request.yml │ ├── 030_check-network.yml │ ├── 040_check-network-adv.yml │ ├── 100_check-k8s-conformance.yml │ ├── roles/ │ │ └── cluster-dump/ │ │ └── tasks/ │ │ └── main.yml │ └── tests.yml ├── upgrade-cluster.yml └── upgrade_cluster.yml
SYMBOL INDEX (52 symbols across 8 files)
FILE: contrib/aws_inventory/kubespray-aws-inventory.py
class SearchEC2Tags (line 9) | class SearchEC2Tags(object):
method __init__ (line 11) | def __init__(self):
method parse_args (line 19) | def parse_args(self):
method search_tags (line 33) | def search_tags(self):
FILE: contrib/offline/upload2artifactory.py
function upload_file (line 21) | def upload_file(file_path, destination_url, username, token):
function upload_files (line 44) | def upload_files(base_url, username, token):
FILE: contrib/terraform/terraform.py
function tfstates (line 34) | def tfstates(root=None):
function convert_to_v3_structure (line 41) | def convert_to_v3_structure(attributes, prefix=''):
function iterresources (line 62) | def iterresources(filenames):
function _clean_dc (line 98) | def _clean_dc(dcname):
function iterhosts (line 104) | def iterhosts(resources):
function iterips (line 116) | def iterips(resources):
function parses (line 124) | def parses(prefix):
function calculate_mantl_vars (line 132) | def calculate_mantl_vars(func):
function _parse_prefix (line 154) | def _parse_prefix(source, prefix, sep='.'):
function parse_attr_list (line 167) | def parse_attr_list(source, prefix, sep='.'):
function parse_dict (line 176) | def parse_dict(source, prefix, sep='.'):
function parse_list (line 180) | def parse_list(source, prefix, sep='.'):
function parse_bool (line 184) | def parse_bool(string_form):
function sanitize_groups (line 197) | def sanitize_groups(groups):
function equinix_metal_device (line 209) | def equinix_metal_device(resource, tfvars=None):
function openstack_floating_ips (line 253) | def openstack_floating_ips(resource):
function openstack_floating_ips (line 261) | def openstack_floating_ips(resource):
function openstack_host (line 267) | def openstack_host(resource, module_name):
function iter_host_ips (line 357) | def iter_host_ips(hosts, ips):
function query_host (line 379) | def query_host(hosts, target):
function query_list (line 387) | def query_list(hosts):
function query_hostfile (line 406) | def query_hostfile(hosts):
function main (line 417) | def main():
FILE: plugins/modules/kube.py
class KubeManager (line 120) | class KubeManager(object):
method __init__ (line 122) | def __init__(self, module):
method _execute (line 153) | def _execute(self, cmd):
method _execute_nofail (line 165) | def _execute_nofail(self, cmd):
method create (line 172) | def create(self, check=True, force=True):
method replace (line 194) | def replace(self, force=True):
method delete (line 214) | def delete(self):
method exists (line 248) | def exists(self):
method stop (line 278) | def stop(self):
function main (line 310) | def main():
FILE: roles/bootstrap_os/molecule/default/tests/test_default.py
function test_python (line 10) | def test_python(host):
FILE: scripts/component_hash_update/src/component_hash_update/download.py
function open_yaml (line 33) | def open_yaml(file: Path):
function download_hash (line 61) | def download_hash(downloads: {str: {str: Any}}) -> None:
function main (line 276) | def main():
FILE: scripts/openstack-cleanup/main.py
function main (line 27) | def main():
function map_if_old (line 84) | def map_if_old(fn, items):
function fn_if_old (line 90) | def fn_if_old(fn, item):
FILE: tests/scripts/md-table/main.py
class Data (line 25) | class Data:
method __init__ (line 26) | def __init__(self):
method set (line 33) | def set(self, container_manager, network_plugin, os):
method exists (line 39) | def exists(self, container_manager, network_plugin, os):
method jinja (line 42) | def jinja(self):
method markdown (line 55) | def markdown(self):
Condensed preview — 1193 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (3,512K chars).
[
{
"path": ".ansible-lint",
"chars": 1390,
"preview": "---\nskip_list:\n # see https://docs.ansible.com/ansible-lint/rules/default_rules.html for a list of all default rules\n\n "
},
{
"path": ".ansible-lint-ignore",
"chars": 514,
"preview": "# This file contains ignores rule violations for ansible-lint\ninventory/sample/group_vars/k8s_cluster/k8s-cluster.yml ji"
},
{
"path": ".editorconfig",
"chars": 285,
"preview": "root = true\n\n[*.{yaml,yml,yml.j2,yaml.j2}]\nindent_style = space\nindent_size = 2\ntrim_trailing_whitespace = true\ninsert_f"
},
{
"path": ".gitattributes",
"chars": 41,
"preview": "docs/_sidebar.md linguist-generated=true\n"
},
{
"path": ".github/ISSUE_TEMPLATE/bug-report.yaml",
"chars": 3981,
"preview": "---\nname: Bug Report\ndescription: Report a bug encountered while using Kubespray\nlabels: kind/bug\nbody:\n - type: markdo"
},
{
"path": ".github/ISSUE_TEMPLATE/config.yml",
"chars": 192,
"preview": "---\nblank_issues_enabled: false\ncontact_links:\n - name: Support Request\n url: https://kubernetes.slack.com/channels/"
},
{
"path": ".github/ISSUE_TEMPLATE/enhancement.yaml",
"chars": 480,
"preview": "---\nname: Enhancement Request\ndescription: Suggest an enhancement to the Kubespray project\nlabels: kind/feature\nbody:\n "
},
{
"path": ".github/ISSUE_TEMPLATE/failing-test.yaml",
"chars": 960,
"preview": "---\nname: Failing Test\ndescription: Report test failures in Kubespray CI jobs\nlabels: kind/failing-test\nbody:\n - type: "
},
{
"path": ".github/PULL_REQUEST_TEMPLATE.md",
"chars": 2136,
"preview": "<!-- Thanks for sending a pull request! Here are some tips for you:\n\n1. If this is your first time, please read our co"
},
{
"path": ".github/dependabot.yml",
"chars": 416,
"preview": "version: 2\nupdates:\n - package-ecosystem: \"pip\"\n directory: \"/\"\n schedule:\n interval: \"weekly\"\n labels:\n "
},
{
"path": ".github/workflows/auto-label-os.yml",
"chars": 835,
"preview": "name: Issue labeler\non:\n issues:\n types: [opened]\n\npermissions:\n contents: read\n\njobs:\n label-component:\n runs-"
},
{
"path": ".github/workflows/upgrade-patch-versions-schedule.yml",
"chars": 1683,
"preview": "name: Upgrade Kubespray components with new patches versions - all branches\n\non:\n schedule:\n - cron: '22 2 * * *' # ev"
},
{
"path": ".github/workflows/upgrade-patch-versions.yml",
"chars": 1243,
"preview": "on:\n workflow_call:\n inputs:\n branch:\n description: Which branch to update with new patch versions\n "
},
{
"path": ".gitignore",
"chars": 1590,
"preview": ".vagrant\n*.retry\n**/vagrant_ansible_inventory\n*.iml\ntemp\ncontrib/offline/container-images\ncontrib/offline/container-imag"
},
{
"path": ".gitlab-ci/build.yml",
"chars": 1117,
"preview": "---\npipeline-image:\n cache:\n key: $CI_COMMIT_REF_SLUG\n paths:\n - image-cache\n tags:\n - ffci\n stage: bui"
},
{
"path": ".gitlab-ci/kubevirt.yml",
"chars": 5259,
"preview": "---\n.kubevirt:\n extends: .job-moderated\n interruptible: true\n script:\n - ansible-playbook tests/cloud_playbooks/cr"
},
{
"path": ".gitlab-ci/lint.yml",
"chars": 588,
"preview": "---\npre-commit:\n stage: test\n tags:\n - ffci\n image: 'ghcr.io/pre-commit-ci/runner-image@sha256:fe01a6ec51b298412990b"
},
{
"path": ".gitlab-ci/molecule.yml",
"chars": 1382,
"preview": "---\n.molecule:\n tags: [ffci]\n rules: # run on ci-short as well\n - if: $CI_COMMIT_BRANCH =~ /^pr-.*$/\n when: on_suc"
},
{
"path": ".gitlab-ci/terraform.yml",
"chars": 3982,
"preview": "---\n# Tests for contrib/terraform/\n.terraform_install:\n extends: .job\n needs:\n - pipeline-image\n variables:\n TF"
},
{
"path": ".gitlab-ci/vagrant.yml",
"chars": 1543,
"preview": "---\nvagrant:\n extends: .job-moderated\n variables:\n CI_PLATFORM: \"vagrant\"\n SSH_USER: \"vagrant\"\n VAGRANT_DEFAU"
},
{
"path": ".gitlab-ci.yml",
"chars": 1760,
"preview": "---\nstages:\n - build # build docker image used in most other jobs\n - test # unit tests\n - deploy-part1 # kubespray ru"
},
{
"path": ".gitmodules",
"chars": 0,
"preview": ""
},
{
"path": ".md_style.rb",
"chars": 73,
"preview": "all\nexclude_rule 'MD013'\nexclude_rule 'MD029'\nrule 'MD007', :indent => 2\n"
},
{
"path": ".mdlrc",
"chars": 47,
"preview": "style \"#{File.dirname(__FILE__)}/.md_style.rb\"\n"
},
{
"path": ".nojekyll",
"chars": 0,
"preview": ""
},
{
"path": ".pre-commit-config.yaml",
"chars": 3015,
"preview": "---\nrepos:\n - repo: https://github.com/pre-commit/pre-commit-hooks\n rev: v6.0.0\n hooks:\n - id: check-added-l"
},
{
"path": ".yamllint",
"chars": 681,
"preview": "---\nextends: default\n\nignore: |\n .git/\n .github/\n # Generated file\n tests/files/custom_cni/cilium.yaml\n# https://ans"
},
{
"path": "CHANGELOG.md",
"chars": 91,
"preview": "# See our release notes on [GitHub](https://github.com/kubernetes-sigs/kubespray/releases)\n"
},
{
"path": "CNAME",
"chars": 13,
"preview": "kubespray.io\n"
},
{
"path": "CONTRIBUTING.md",
"chars": 2371,
"preview": "# Contributing guidelines\n\n## How to become a contributor and submit your own code\n\n### Environment setup\n\nIt is recomme"
},
{
"path": "Dockerfile",
"chars": 1754,
"preview": "# syntax=docker/dockerfile:1\n\n# Use immutable image tags rather than mutable tags (like ubuntu:24.04)\nFROM ubuntu:noble-"
},
{
"path": "LICENSE",
"chars": 11339,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "OWNERS",
"chars": 174,
"preview": "# See the OWNERS docs at https://go.k8s.io/owners\n\napprovers:\n - kubespray-approvers\nreviewers:\n - kubespray-reviewers"
},
{
"path": "OWNERS_ALIASES",
"chars": 408,
"preview": "aliases:\n kubespray-approvers:\n - ant31\n - mzaian\n - tico88612\n - vannten\n - yankay\n kubespray-reviewer"
},
{
"path": "README.md",
"chars": 11867,
"preview": "# Deploy a Production Ready Kubernetes Cluster\n\n![Kubernetes Logo](https://raw.githubusercontent.com/kubernetes-sigs/kub"
},
{
"path": "RELEASE.md",
"chars": 5376,
"preview": "# Release Process\n\nThe Kubespray Project is released on an as-needed basis. The process is as follows:\n\n1. An issue is p"
},
{
"path": "SECURITY_CONTACTS",
"chars": 546,
"preview": "# Defined below are the security contacts for this repo.\n#\n# They are the contact point for the Product Security Committ"
},
{
"path": "Vagrantfile",
"chars": 14858,
"preview": "# -*- mode: ruby -*-\n# # vi: set ft=ruby :\n\n# For help on using kubespray with vagrant, check out docs/developers/vagran"
},
{
"path": "_config.yml",
"chars": 30,
"preview": "---\ntheme: jekyll-theme-slate\n"
},
{
"path": "ansible.cfg",
"chars": 873,
"preview": "[ssh_connection]\npipelining=True\nssh_args = -o ControlMaster=auto -o ControlPersist=30m -o ConnectionAttempts=100 -o Use"
},
{
"path": "cluster.yml",
"chars": 88,
"preview": "---\n- name: Install Kubernetes\n ansible.builtin.import_playbook: playbooks/cluster.yml\n"
},
{
"path": "code-of-conduct.md",
"chars": 148,
"preview": "# Kubernetes Community Code of Conduct\n\nPlease refer to our [Kubernetes Community Code of Conduct](https://git.k8s.io/co"
},
{
"path": "contrib/aws_iam/kubernetes-master-policy.json",
"chars": 465,
"preview": "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Action\": [\"ec2:*\"],\n \"Resource\""
},
{
"path": "contrib/aws_iam/kubernetes-master-role.json",
"chars": 176,
"preview": "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazon"
},
{
"path": "contrib/aws_iam/kubernetes-minion-policy.json",
"chars": 885,
"preview": "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Action\": \"s3:*\",\n \"Resource\": ["
},
{
"path": "contrib/aws_iam/kubernetes-minion-role.json",
"chars": 176,
"preview": "{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": { \"Service\": \"ec2.amazon"
},
{
"path": "contrib/aws_inventory/kubespray-aws-inventory.py",
"chars": 2985,
"preview": "#!/usr/bin/env python\n\nfrom __future__ import print_function\nimport boto3\nimport os\nimport argparse\nimport json\n\nclass S"
},
{
"path": "contrib/aws_inventory/requirements.txt",
"chars": 20,
"preview": "boto3 # Apache-2.0\n"
},
{
"path": "contrib/azurerm/.gitignore",
"chars": 22,
"preview": ".generated\n/inventory\n"
},
{
"path": "contrib/azurerm/README.md",
"chars": 2707,
"preview": "# Kubernetes on Azure with Azure Resource Group Templates\n\nProvision the base infrastructure for a Kubernetes cluster by"
},
{
"path": "contrib/azurerm/apply-rg.sh",
"chars": 772,
"preview": "#!/usr/bin/env bash\n\nset -e\n\nAZURE_RESOURCE_GROUP=\"$1\"\n\nif [ \"$AZURE_RESOURCE_GROUP\" == \"\" ]; then\n echo \"AZURE_RESOU"
},
{
"path": "contrib/azurerm/clear-rg.sh",
"chars": 311,
"preview": "#!/usr/bin/env bash\n\nset -e\n\nAZURE_RESOURCE_GROUP=\"$1\"\n\nif [ \"$AZURE_RESOURCE_GROUP\" == \"\" ]; then\n echo \"AZURE_RESOU"
},
{
"path": "contrib/azurerm/generate-inventory.sh",
"chars": 489,
"preview": "#!/usr/bin/env bash\n\nset -e\n\nAZURE_RESOURCE_GROUP=\"$1\"\n\nif [ \"$AZURE_RESOURCE_GROUP\" == \"\" ]; then\n echo \"AZURE_RESOU"
},
{
"path": "contrib/azurerm/generate-inventory.yml",
"chars": 112,
"preview": "---\n- name: Generate Azure inventory\n hosts: localhost\n gather_facts: false\n roles:\n - generate-inventory\n"
},
{
"path": "contrib/azurerm/generate-inventory_2.yml",
"chars": 114,
"preview": "---\n- name: Generate Azure inventory\n hosts: localhost\n gather_facts: false\n roles:\n - generate-inventory_2\n"
},
{
"path": "contrib/azurerm/generate-templates.yml",
"chars": 112,
"preview": "---\n- name: Generate Azure templates\n hosts: localhost\n gather_facts: false\n roles:\n - generate-templates\n"
},
{
"path": "contrib/azurerm/group_vars/all",
"chars": 2173,
"preview": "\n# Due to some Azure limitations (ex:- Storage Account's name must be unique),\n# this name must be globally unique - it "
},
{
"path": "contrib/azurerm/roles/generate-inventory/tasks/main.yml",
"chars": 316,
"preview": "---\n\n- name: Query Azure VMs\n command: azure vm list-ip-address --json {{ azure_resource_group }}\n register: vm_list_c"
},
{
"path": "contrib/azurerm/roles/generate-inventory/templates/inventory.j2",
"chars": 869,
"preview": "\n{% for vm in vm_list %}\n{% if not use_bastion or vm.name == 'bastion' %}\n{{ vm.name }} ansible_ssh_host={{ vm.networkPr"
},
{
"path": "contrib/azurerm/roles/generate-inventory_2/tasks/main.yml",
"chars": 925,
"preview": "---\n\n- name: Query Azure VMs IPs\n command: az vm list-ip-addresses -o json --resource-group {{ azure_resource_group }}\n"
},
{
"path": "contrib/azurerm/roles/generate-inventory_2/templates/inventory.j2",
"chars": 816,
"preview": "\n{% for vm in vm_ip_list %}\n{% if not use_bastion or vm.virtualMachine.name == 'bastion' %}\n{{ vm.virtualMachine.name }}"
},
{
"path": "contrib/azurerm/roles/generate-inventory_2/templates/loadbalancer_vars.j2",
"chars": 253,
"preview": "## External LB example config\napiserver_loadbalancer_domain_name: {{ lb_pubip.dnsSettings.fqdn }}\nloadbalancer_apiserver"
},
{
"path": "contrib/azurerm/roles/generate-templates/defaults/main.yml",
"chars": 1114,
"preview": "---\napiVersion: \"2015-06-15\"\n\nvirtualNetworkName: \"{{ azure_virtual_network_name | default('KubeVNET') }}\"\n\nsubnetAdminN"
},
{
"path": "contrib/azurerm/roles/generate-templates/tasks/main.yml",
"chars": 485,
"preview": "---\n- name: Set base_dir\n set_fact:\n base_dir: \"{{ playbook_dir }}/.generated/\"\n\n- name: Create base_dir\n file:\n "
},
{
"path": "contrib/azurerm/roles/generate-templates/templates/availability-sets.json",
"chars": 888,
"preview": "{\n \"$schema\": \"https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#\",\n \"contentVersion\": \"1"
},
{
"path": "contrib/azurerm/roles/generate-templates/templates/bastion.json",
"chars": 3324,
"preview": "{\n \"$schema\": \"https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#\",\n \"contentVersion\": \"1"
},
{
"path": "contrib/azurerm/roles/generate-templates/templates/clear-rg.json",
"chars": 205,
"preview": "{\n \"$schema\": \"https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#\",\n \"contentVersion\": \"1"
},
{
"path": "contrib/azurerm/roles/generate-templates/templates/masters.json",
"chars": 6604,
"preview": "{\n \"$schema\": \"https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#\",\n \"contentVersion\": \"1"
},
{
"path": "contrib/azurerm/roles/generate-templates/templates/minions.json",
"chars": 3756,
"preview": "{\n \"$schema\": \"https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#\",\n \"contentVersion\": \"1"
},
{
"path": "contrib/azurerm/roles/generate-templates/templates/network.json",
"chars": 3239,
"preview": "{\n \"$schema\": \"https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#\",\n \"contentVersion\": \"1"
},
{
"path": "contrib/azurerm/roles/generate-templates/templates/storage.json",
"chars": 464,
"preview": "{\n \"$schema\": \"https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#\",\n \"contentVersion\": \"1"
},
{
"path": "contrib/collection.sh",
"chars": 415,
"preview": "#!/bin/bash -eux\n# Install collection from source assuming dependencies are present.\n# Run in SemaphoreUI this bash scri"
},
{
"path": "contrib/offline/README.md",
"chars": 2888,
"preview": "# Offline deployment\n\n## manage-offline-container-images.sh\n\nContainer image collecting script for offline deployment\n\nT"
},
{
"path": "contrib/offline/docker-daemon.json",
"chars": 44,
"preview": "{ \"insecure-registries\":[\"HOSTNAME:5000\"] }\n"
},
{
"path": "contrib/offline/generate_list.sh",
"chars": 1329,
"preview": "#!/bin/bash\nset -eo pipefail\n\nCURRENT_DIR=$(cd $(dirname $0); pwd)\nTEMP_DIR=\"${CURRENT_DIR}/temp\"\nREPO_ROOT_DIR=\"${CURRE"
},
{
"path": "contrib/offline/generate_list.yml",
"chars": 575,
"preview": "---\n- name: Collect container images for offline deployment\n hosts: localhost\n become: false\n\n roles:\n # Just load"
},
{
"path": "contrib/offline/manage-offline-container-images.sh",
"chars": 7606,
"preview": "#!/usr/bin/env bash\n\nOPTION=$1\nCURRENT_DIR=$(cd $(dirname $0); pwd)\nTEMP_DIR=\"${CURRENT_DIR}/temp\"\n\nIMAGE_TAR_FILE=\"${CU"
},
{
"path": "contrib/offline/manage-offline-files.sh",
"chars": 1453,
"preview": "#!/bin/bash\n\nCURRENT_DIR=$( dirname \"$(readlink -f \"$0\")\" )\nOFFLINE_FILES_DIR_NAME=\"offline-files\"\nOFFLINE_FILES_DIR=\"${"
},
{
"path": "contrib/offline/nginx.conf",
"chars": 1186,
"preview": "user nginx;\nworker_processes auto;\nerror_log /var/log/nginx/error.log;\npid /run/nginx.pid;\ninclude /usr/share/nginx/modu"
},
{
"path": "contrib/offline/registries.conf",
"chars": 189,
"preview": "[registries.search]\nregistries = ['registry.access.redhat.com', 'registry.redhat.io', 'docker.io']\n\n[registries.insecure"
},
{
"path": "contrib/offline/upload2artifactory.py",
"chars": 2469,
"preview": "#!/usr/bin/env python3\n\"\"\"This is a helper script to manage-offline-files.sh.\n\nAfter running manage-offline-files.sh, yo"
},
{
"path": "contrib/os-services/os-services.yml",
"chars": 80,
"preview": "---\n- name: Disable firewalld/ufw\n hosts: all\n roles:\n - { role: prepare }\n"
},
{
"path": "contrib/os-services/roles/prepare/defaults/main.yml",
"chars": 36,
"preview": "---\ndisable_service_firewall: false\n"
},
{
"path": "contrib/os-services/roles/prepare/tasks/main.yml",
"chars": 599,
"preview": "---\n- name: Disable firewalld and ufw\n when:\n - disable_service_firewall is defined and disable_service_firewall\n blo"
},
{
"path": "contrib/terraform/aws/.gitignore",
"chars": 42,
"preview": "*.tfstate*\n.terraform.lock.hcl\n.terraform\n"
},
{
"path": "contrib/terraform/aws/README.md",
"chars": 4953,
"preview": "# Kubernetes on AWS with Terraform\n\n## Overview\n\nThis project will create:\n\n- VPC with Public and Private Subnets in # A"
},
{
"path": "contrib/terraform/aws/create-infrastructure.tf",
"chars": 5958,
"preview": "terraform {\n required_version = \">= 0.12.0\"\n required_providers {\n aws = {\n source = \"hashicorp/aws\"\n ve"
},
{
"path": "contrib/terraform/aws/credentials.tfvars.example",
"chars": 170,
"preview": "#AWS Access Key\nAWS_ACCESS_KEY_ID = \"\"\n#AWS Secret Key\nAWS_SECRET_ACCESS_KEY = \"\"\n#EC2 SSH Key Name\nAWS_SSH_KEY_NAME = \""
},
{
"path": "contrib/terraform/aws/modules/iam/main.tf",
"chars": 2906,
"preview": "#Add AWS Roles for Kubernetes\n\nresource \"aws_iam_role\" \"kube_control_plane\" {\n name = \"kubernetes-${var.aws_cluster_nam"
},
{
"path": "contrib/terraform/aws/modules/iam/outputs.tf",
"chars": 185,
"preview": "output \"kube_control_plane-profile\" {\n value = aws_iam_instance_profile.kube_control_plane.name\n}\n\noutput \"kube-worker-"
},
{
"path": "contrib/terraform/aws/modules/iam/variables.tf",
"chars": 66,
"preview": "variable \"aws_cluster_name\" {\n description = \"Name of Cluster\"\n}\n"
},
{
"path": "contrib/terraform/aws/modules/nlb/main.tf",
"chars": 1375,
"preview": "# Create a new AWS NLB for K8S API\nresource \"aws_lb\" \"aws-nlb-api\" {\n name = \"kubernetes-nl"
},
{
"path": "contrib/terraform/aws/modules/nlb/outputs.tf",
"chars": 211,
"preview": "output \"aws_nlb_api_id\" {\n value = aws_lb.aws-nlb-api.id\n}\n\noutput \"aws_nlb_api_fqdn\" {\n value = aws_lb.aws-nlb-api.dn"
},
{
"path": "contrib/terraform/aws/modules/nlb/variables.tf",
"chars": 582,
"preview": "variable \"aws_cluster_name\" {\n description = \"Name of Cluster\"\n}\n\nvariable \"aws_vpc_id\" {\n description = \"AWS VPC ID\"\n"
},
{
"path": "contrib/terraform/aws/modules/vpc/main.tf",
"chars": 4416,
"preview": "resource \"aws_vpc\" \"cluster-vpc\" {\n cidr_block = var.aws_vpc_cidr_block\n\n #DNS Related Entries\n enable_dns_support "
},
{
"path": "contrib/terraform/aws/modules/vpc/outputs.tf",
"chars": 369,
"preview": "output \"aws_vpc_id\" {\n value = aws_vpc.cluster-vpc.id\n}\n\noutput \"aws_subnet_ids_private\" {\n value = aws_subnet.cluster"
},
{
"path": "contrib/terraform/aws/modules/vpc/variables.tf",
"chars": 638,
"preview": "variable \"aws_vpc_cidr_block\" {\n description = \"CIDR Blocks for AWS VPC\"\n}\n\nvariable \"aws_cluster_name\" {\n description"
},
{
"path": "contrib/terraform/aws/output.tf",
"chars": 622,
"preview": "output \"bastion_ip\" {\n value = join(\"\\n\", aws_instance.bastion-server.*.public_ip)\n}\n\noutput \"masters\" {\n value = join"
},
{
"path": "contrib/terraform/aws/sample-inventory/cluster.tfvars",
"chars": 945,
"preview": "#Global Vars\naws_cluster_name = \"devtest\"\n\n#VPC Vars\naws_vpc_cidr_block = \"10.250.192.0/18\"\n\naws_cidr_subnets_private = "
},
{
"path": "contrib/terraform/aws/templates/inventory.tpl",
"chars": 354,
"preview": "[all]\n${connection_strings_master}\n${connection_strings_node}\n${connection_strings_etcd}\n${public_ip_address_bastion}\n\n["
},
{
"path": "contrib/terraform/aws/terraform.tfvars",
"chars": 1132,
"preview": "#Global Vars\naws_cluster_name = \"devtest\"\n\n#VPC Vars\naws_vpc_cidr_block = \"10.250.192.0/18\"\naws_cidr_subnets_priva"
},
{
"path": "contrib/terraform/aws/terraform.tfvars.example",
"chars": 745,
"preview": "#Global Vars\naws_cluster_name = \"devtest\"\n\n#VPC Vars\naws_vpc_cidr_block = \"10.250.192.0/18\"\naws_cidr_subnets_private = ["
},
{
"path": "contrib/terraform/aws/variables.tf",
"chars": 2937,
"preview": "variable \"AWS_ACCESS_KEY_ID\" {\n description = \"AWS Access Key\"\n}\n\nvariable \"AWS_SECRET_ACCESS_KEY\" {\n description = \"A"
},
{
"path": "contrib/terraform/exoscale/README.md",
"chars": 5651,
"preview": "# Kubernetes on Exoscale with Terraform\n\nProvision a Kubernetes cluster on [Exoscale](https://www.exoscale.com/) using T"
},
{
"path": "contrib/terraform/exoscale/default.tfvars",
"chars": 1402,
"preview": "prefix = \"default\"\nzone = \"ch-gva-2\"\n\ninventory_file = \"inventory.ini\"\n\nssh_public_keys = [\n # Put your public SSH ke"
},
{
"path": "contrib/terraform/exoscale/main.tf",
"chars": 1620,
"preview": "provider \"exoscale\" {}\n\nmodule \"kubernetes\" {\n source = \"./modules/kubernetes-cluster\"\n\n prefix = var.prefix\n zone "
},
{
"path": "contrib/terraform/exoscale/modules/kubernetes-cluster/main.tf",
"chars": 5556,
"preview": "data \"exoscale_template\" \"os_image\" {\n for_each = var.machines\n\n zone = var.zone\n name = each.value.boot_disk.image_n"
},
{
"path": "contrib/terraform/exoscale/modules/kubernetes-cluster/output.tf",
"chars": 1053,
"preview": "output \"master_ip_addresses\" {\n value = {\n for key, instance in exoscale_compute_instance.master :\n instance.name"
},
{
"path": "contrib/terraform/exoscale/modules/kubernetes-cluster/templates/cloud-init.tmpl",
"chars": 1811,
"preview": "#cloud-config\n%{ if ceph_partition_size > 0 || node_local_partition_size > 0}\nbootcmd:\n- [ cloud-init-per, once, move-se"
},
{
"path": "contrib/terraform/exoscale/modules/kubernetes-cluster/variables.tf",
"chars": 853,
"preview": "variable \"zone\" {\n type = string\n # This is currently the only zone that is supposed to be supporting\n # so called \"m"
},
{
"path": "contrib/terraform/exoscale/modules/kubernetes-cluster/versions.tf",
"chars": 157,
"preview": "terraform {\n required_providers {\n exoscale = {\n source = \"exoscale/exoscale\"\n version = \">= 0.21\"\n }\n"
},
{
"path": "contrib/terraform/exoscale/output.tf",
"chars": 351,
"preview": "output \"master_ips\" {\n value = module.kubernetes.master_ip_addresses\n}\n\noutput \"worker_ips\" {\n value = module.kubernet"
},
{
"path": "contrib/terraform/exoscale/sample-inventory/cluster.tfvars",
"chars": 1365,
"preview": "prefix = \"default\"\nzone = \"ch-gva-2\"\n\ninventory_file = \"inventory.ini\"\n\nssh_public_keys = [\n # Put your public SSH ke"
},
{
"path": "contrib/terraform/exoscale/templates/inventory.tpl",
"chars": 297,
"preview": "[all]\n${connection_strings_master}\n${connection_strings_worker}\n\n[kube_control_plane]\n${list_master}\n\n[kube_control_plan"
},
{
"path": "contrib/terraform/exoscale/variables.tf",
"chars": 1140,
"preview": "variable \"zone\" {\n description = \"The zone where to run the cluster\"\n}\n\nvariable \"prefix\" {\n description = \"Prefix for"
},
{
"path": "contrib/terraform/exoscale/versions.tf",
"chars": 267,
"preview": "terraform {\n required_providers {\n exoscale = {\n source = \"exoscale/exoscale\"\n version = \">= 0.21\"\n }\n"
},
{
"path": "contrib/terraform/gcp/README.md",
"chars": 5079,
"preview": "# Kubernetes on GCP with Terraform\n\nProvision a Kubernetes cluster on GCP using Terraform and Kubespray\n\n## Overview\n\nTh"
},
{
"path": "contrib/terraform/gcp/generate-inventory.sh",
"chars": 2093,
"preview": "#!/bin/bash\n\n#\n# Generates a inventory file based on the terraform output.\n# After provisioning a cluster, simply run th"
},
{
"path": "contrib/terraform/gcp/main.tf",
"chars": 1064,
"preview": "terraform {\n required_providers {\n google = {\n source = \"hashicorp/google\"\n version = \"~> 4.0\"\n }\n }\n"
},
{
"path": "contrib/terraform/gcp/modules/kubernetes-cluster/main.tf",
"chars": 9435,
"preview": "#################################################\n##\n## General\n##\n\nresource \"google_compute_network\" \"main\" {\n name = "
},
{
"path": "contrib/terraform/gcp/modules/kubernetes-cluster/output.tf",
"chars": 832,
"preview": "output \"master_ip_addresses\" {\n value = {\n for key, instance in google_compute_instance.master :\n instance.name ="
},
{
"path": "contrib/terraform/gcp/modules/kubernetes-cluster/variables.tf",
"chars": 1303,
"preview": "variable \"region\" {\n type = string\n}\n\nvariable \"prefix\" {}\n\nvariable \"machines\" {\n type = map(object({\n node_type ="
},
{
"path": "contrib/terraform/gcp/output.tf",
"chars": 351,
"preview": "output \"master_ips\" {\n value = module.kubernetes.master_ip_addresses\n}\n\noutput \"worker_ips\" {\n value = module.kubernet"
},
{
"path": "contrib/terraform/gcp/tfvars.json",
"chars": 1309,
"preview": "{\n \"gcp_project_id\": \"GCP_PROJECT_ID\",\n \"region\": \"us-central1\",\n \"ssh_pub_key\": \"~/.ssh/id_rsa.pub\",\n\n \"keyfile_loc"
},
{
"path": "contrib/terraform/gcp/variables.tf",
"chars": 1955,
"preview": "variable keyfile_location {\n description = \"Location of the json keyfile to use with the google provider\"\n type "
},
{
"path": "contrib/terraform/hetzner/README.md",
"chars": 3973,
"preview": "# Kubernetes on Hetzner with Terraform\n\nProvision a Kubernetes cluster on [Hetzner](https://www.hetzner.com/cloud) using"
},
{
"path": "contrib/terraform/hetzner/default.tfvars",
"chars": 760,
"preview": "prefix = \"default\"\nzone = \"hel1\"\nnetwork_zone = \"eu-central\"\ninventory_file = \"inventory.ini\"\n\nssh_p"
},
{
"path": "contrib/terraform/hetzner/main.tf",
"chars": 1754,
"preview": "provider \"hcloud\" {}\n\nmodule \"kubernetes\" {\n source = \"./modules/kubernetes-cluster\"\n # source = \"./modules/kubernetes"
},
{
"path": "contrib/terraform/hetzner/modules/kubernetes-cluster/main.tf",
"chars": 2476,
"preview": "resource \"hcloud_network\" \"kubernetes\" {\n name = \"${var.prefix}-network\"\n ip_range = var.private_network_cidr\n}\n\nr"
},
{
"path": "contrib/terraform/hetzner/modules/kubernetes-cluster/output.tf",
"chars": 630,
"preview": "output \"master_ip_addresses\" {\n value = {\n for key, instance in hcloud_server.master :\n instance.name => {\n "
},
{
"path": "contrib/terraform/hetzner/modules/kubernetes-cluster/templates/cloud-init.tmpl",
"chars": 324,
"preview": "#cloud-config\n\nusers:\n - default\n - name: ubuntu\n shell: /bin/bash\n sudo: \"ALL=(ALL) NOPASSWD:ALL\"\n ssh_autho"
},
{
"path": "contrib/terraform/hetzner/modules/kubernetes-cluster/variables.tf",
"chars": 637,
"preview": "variable \"zone\" {\n type = string\n}\n\nvariable \"prefix\" {}\n\nvariable \"machines\" {\n type = map(object({\n node_type = s"
},
{
"path": "contrib/terraform/hetzner/modules/kubernetes-cluster/versions.tf",
"chars": 156,
"preview": "terraform {\n required_providers {\n hcloud = {\n source = \"hetznercloud/hcloud\"\n version = \"1.38.2\"\n }\n "
},
{
"path": "contrib/terraform/hetzner/modules/kubernetes-cluster-flatcar/main.tf",
"chars": 3386,
"preview": "resource \"hcloud_network\" \"kubernetes\" {\n name = \"${var.prefix}-network\"\n ip_range = var.private_network_cidr\n}\n\nr"
},
{
"path": "contrib/terraform/hetzner/modules/kubernetes-cluster-flatcar/outputs.tf",
"chars": 698,
"preview": "output \"master_ip_addresses\" {\n value = {\n for name, machine in var.machines :\n name => {\n \"private_ip\" "
},
{
"path": "contrib/terraform/hetzner/modules/kubernetes-cluster-flatcar/templates/machine.yaml.tmpl",
"chars": 389,
"preview": "variant: flatcar\nversion: 1.0.0\n\npasswd:\n users:\n - name: ${user_flatcar}\n ssh_authorized_keys: ${ssh_keys}\n\nst"
},
{
"path": "contrib/terraform/hetzner/modules/kubernetes-cluster-flatcar/variables.tf",
"chars": 832,
"preview": "\nvariable \"zone\" {\n type = string\n default = \"fsn1\"\n}\n\nvariable \"prefix\" {\n default = \"k8s\"\n}\n\nvariable \"user_flat"
},
{
"path": "contrib/terraform/hetzner/modules/kubernetes-cluster-flatcar/versions.tf",
"chars": 222,
"preview": "terraform {\n required_providers {\n hcloud = {\n source = \"hetznercloud/hcloud\"\n }\n ct = {\n source = "
},
{
"path": "contrib/terraform/hetzner/output.tf",
"chars": 145,
"preview": "output \"master_ips\" {\n value = module.kubernetes.master_ip_addresses\n}\n\noutput \"worker_ips\" {\n value = module.kubernet"
},
{
"path": "contrib/terraform/hetzner/sample-inventory/cluster.tfvars",
"chars": 760,
"preview": "prefix = \"default\"\nzone = \"hel1\"\nnetwork_zone = \"eu-central\"\ninventory_file = \"inventory.ini\"\n\nssh_p"
},
{
"path": "contrib/terraform/hetzner/templates/inventory.tpl",
"chars": 250,
"preview": "[all]\n${connection_strings_master}\n${connection_strings_worker}\n\n[kube_control_plane]\n${list_master}\n\n[etcd]\n${list_mast"
},
{
"path": "contrib/terraform/hetzner/variables.tf",
"chars": 1359,
"preview": "variable \"zone\" {\n description = \"The zone where to run the cluster\"\n}\nvariable \"network_zone\" {\n description = \"The n"
},
{
"path": "contrib/terraform/hetzner/versions.tf",
"chars": 207,
"preview": "terraform {\n required_providers {\n hcloud = {\n source = \"hetznercloud/hcloud\"\n version = \"1.38.2\"\n }\n "
},
{
"path": "contrib/terraform/openstack/.gitignore",
"chars": 80,
"preview": ".terraform\n*.tfvars\n!sample-inventory/cluster.tfvars\n*.tfstate\n*.tfstate.backup\n"
},
{
"path": "contrib/terraform/openstack/README.md",
"chars": 33987,
"preview": "# Kubernetes on OpenStack with Terraform\n\nProvision a Kubernetes cluster with [Terraform](https://www.terraform.io) on\nO"
},
{
"path": "contrib/terraform/openstack/kubespray.tf",
"chars": 8134,
"preview": "module \"network\" {\n source = \"./modules/network\"\n\n external_net = var.external_net\n network_name = "
},
{
"path": "contrib/terraform/openstack/modules/compute/ansible_bastion_template.txt",
"chars": 239,
"preview": "ansible_ssh_common_args: \"-o ProxyCommand='ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -W %h:%p -q U"
},
{
"path": "contrib/terraform/openstack/modules/compute/main.tf",
"chars": 44810,
"preview": "data \"openstack_images_image_v2\" \"vm_image\" {\n count = var.image_uuid == \"\" ? 1 : 0\n most_recent = true\n name = var.i"
},
{
"path": "contrib/terraform/openstack/modules/compute/outputs.tf",
"chars": 171,
"preview": "output \"k8s_master_ips\" {\n value = concat(openstack_compute_instance_v2.k8s_master_no_floating_ip.*, openstack_compute_"
},
{
"path": "contrib/terraform/openstack/modules/compute/templates/cloudinit.yaml.tmpl",
"chars": 1659,
"preview": "%{~ if length(extra_partitions) > 0 || netplan_critical_dhcp_interface != \"\" }\n#cloud-config\nbootcmd:\n%{~ for idx, parti"
},
{
"path": "contrib/terraform/openstack/modules/compute/variables.tf",
"chars": 4624,
"preview": "variable \"cluster_name\" {}\n\nvariable \"az_list\" {\n type = list(string)\n}\n\nvariable \"az_list_node\" {\n type = list(string"
},
{
"path": "contrib/terraform/openstack/modules/compute/versions.tf",
"chars": 153,
"preview": "terraform {\n required_providers {\n openstack = {\n source = \"terraform-provider-openstack/openstack\"\n }\n }\n "
},
{
"path": "contrib/terraform/openstack/modules/ips/main.tf",
"chars": 1905,
"preview": "resource \"null_resource\" \"dummy_dependency\" {\n triggers = {\n dependency_id = var.router_id\n }\n depends_on = [\n "
},
{
"path": "contrib/terraform/openstack/modules/ips/outputs.tf",
"chars": 1707,
"preview": "locals {\n k8s_masters_reserved_fips = {\n for key, value in var.k8s_masters : key => {\n address = value.reserved"
},
{
"path": "contrib/terraform/openstack/modules/ips/variables.tf",
"chars": 428,
"preview": "variable \"number_of_k8s_masters\" {}\n\nvariable \"number_of_k8s_masters_no_etcd\" {}\n\nvariable \"number_of_k8s_nodes\" {}\n\nvar"
},
{
"path": "contrib/terraform/openstack/modules/ips/versions.tf",
"chars": 206,
"preview": "terraform {\n required_providers {\n null = {\n source = \"hashicorp/null\"\n }\n openstack = {\n source = \""
},
{
"path": "contrib/terraform/openstack/modules/loadbalancer/main.tf",
"chars": 2277,
"preview": "resource \"openstack_lb_loadbalancer_v2\" \"k8s_lb\" {\n count = var.k8s_master_loadbalancer_enabled ? 1 : 0\n n"
},
{
"path": "contrib/terraform/openstack/modules/loadbalancer/variables.tf",
"chars": 313,
"preview": "variable \"cluster_name\" {}\n\nvariable \"subnet_id\" {}\n\nvariable \"floatingip_pool\" {}\n\nvariable \"k8s_master_ips\" {}\n\nvariab"
},
{
"path": "contrib/terraform/openstack/modules/loadbalancer/versions.tf",
"chars": 155,
"preview": "terraform {\n required_providers {\n openstack = {\n source = \"terraform-provider-openstack/openstack\"\n }\n }\n "
},
{
"path": "contrib/terraform/openstack/modules/network/main.tf",
"chars": 1336,
"preview": "resource \"openstack_networking_router_v2\" \"k8s\" {\n name = \"${var.cluster_name}-router\"\n count "
},
{
"path": "contrib/terraform/openstack/modules/network/outputs.tf",
"chars": 513,
"preview": "output \"router_id\" {\n value = \"%{if var.use_neutron == 1} ${var.router_id == null ? element(concat(openstack_networking"
},
{
"path": "contrib/terraform/openstack/modules/network/variables.tf",
"chars": 294,
"preview": "variable \"external_net\" {}\n\nvariable \"network_name\" {}\n\nvariable \"network_dns_domain\" {}\n\nvariable \"cluster_name\" {}\n\nva"
},
{
"path": "contrib/terraform/openstack/modules/network/versions.tf",
"chars": 155,
"preview": "terraform {\n required_providers {\n openstack = {\n source = \"terraform-provider-openstack/openstack\"\n }\n }\n "
},
{
"path": "contrib/terraform/openstack/variables.tf",
"chars": 8980,
"preview": "variable \"cluster_name\" {\n default = \"example\"\n}\n\nvariable \"az_list\" {\n description = \"List of Availability Zones to u"
},
{
"path": "contrib/terraform/openstack/versions.tf",
"chars": 180,
"preview": "terraform {\n required_providers {\n openstack = {\n source = \"terraform-provider-openstack/openstack\"\n vers"
},
{
"path": "contrib/terraform/terraform.py",
"chars": 16061,
"preview": "#!/usr/bin/env python3\n#\n# Copyright 2015 Cisco Systems, Inc.\n#\n# Licensed under the Apache License, Version 2.0 (the \"L"
},
{
"path": "contrib/terraform/upcloud/README.md",
"chars": 10185,
"preview": "# Kubernetes on UpCloud with Terraform\n\nProvision a Kubernetes cluster on [UpCloud](https://upcloud.com/) using Terrafor"
},
{
"path": "contrib/terraform/upcloud/cluster-settings.tfvars",
"chars": 4306,
"preview": "# See: https://developers.upcloud.com/1.3/5-zones/\nzone = \"fi-hel1\"\nprivate_cloud = false\n\n# Only used if priva"
},
{
"path": "contrib/terraform/upcloud/main.tf",
"chars": 1944,
"preview": "\nterraform {\n required_version = \">= 0.13.0\"\n}\nprovider \"upcloud\" {\n # Your UpCloud credentials are read from environ"
},
{
"path": "contrib/terraform/upcloud/modules/kubernetes-cluster/main.tf",
"chars": 26190,
"preview": "locals {\n # Create a list of all disks to create\n disks = flatten([\n for node_name, machine in var.machines : [\n "
},
{
"path": "contrib/terraform/upcloud/modules/kubernetes-cluster/output.tf",
"chars": 265,
"preview": "output \"master_ip\" {\n value = local.master_ip\n}\n\noutput \"worker_ip\" {\n value = local.worker_ip\n}\n\noutput \"bastion_ip\" "
},
{
"path": "contrib/terraform/upcloud/modules/kubernetes-cluster/variables.tf",
"chars": 4107,
"preview": "variable \"prefix\" {\n type = string\n}\n\nvariable \"zone\" {\n type = string\n}\n\nvariable \"private_cloud\" {\n type = bool\n}\n\n"
},
{
"path": "contrib/terraform/upcloud/modules/kubernetes-cluster/versions.tf",
"chars": 159,
"preview": "\nterraform {\n required_providers {\n upcloud = {\n source = \"UpCloudLtd/upcloud\"\n version = \"~>5.29.1\"\n "
},
{
"path": "contrib/terraform/upcloud/output.tf",
"chars": 270,
"preview": "\noutput \"master_ip\" {\n value = module.kubernetes.master_ip\n}\n\noutput \"worker_ip\" {\n value = module.kubernetes.worker_i"
},
{
"path": "contrib/terraform/upcloud/sample-inventory/cluster.tfvars",
"chars": 3171,
"preview": "# See: https://developers.upcloud.com/1.3/5-zones/\nzone = \"fi-hel1\"\nusername = \"ubuntu\"\n\n# Prefix to use for all res"
},
{
"path": "contrib/terraform/upcloud/templates/inventory.tpl",
"chars": 722,
"preview": "[all]\n%{ for name, ips in master_ip ~}\n${name} ansible_user=${username} ansible_host=${lookup(ips, \"public\", ips.private"
},
{
"path": "contrib/terraform/upcloud/variables.tf",
"chars": 6476,
"preview": "variable \"prefix\" {\n type = string\n default = \"kubespray\"\n\n description = \"Prefix that is used to distinguish thes"
},
{
"path": "contrib/terraform/upcloud/versions.tf",
"chars": 159,
"preview": "\nterraform {\n required_providers {\n upcloud = {\n source = \"UpCloudLtd/upcloud\"\n version = \"~>5.29.1\"\n "
},
{
"path": "contrib/terraform/vsphere/README.md",
"chars": 4693,
"preview": "# Kubernetes on vSphere with Terraform\n\nProvision a Kubernetes cluster on [vSphere](https://www.vmware.com/products/vsph"
},
{
"path": "contrib/terraform/vsphere/default.tfvars",
"chars": 1010,
"preview": "prefix = \"k8s\"\n\ninventory_file = \"inventory.ini\"\n\nnetwork = \"VM Network\"\n\nmachines = {\n \"master-0\" : {\n \"node_type\" "
},
{
"path": "contrib/terraform/vsphere/main.tf",
"chars": 3126,
"preview": "provider \"vsphere\" {\n # Username and password set through env vars VSPHERE_USER and VSPHERE_PASSWORD\n user = var.v"
},
{
"path": "contrib/terraform/vsphere/modules/kubernetes-cluster/main.tf",
"chars": 4379,
"preview": "resource \"vsphere_virtual_machine\" \"worker\" {\n for_each = {\n for name, machine in var.machines :\n name => machine"
},
{
"path": "contrib/terraform/vsphere/modules/kubernetes-cluster/output.tf",
"chars": 318,
"preview": "output \"master_ip\" {\n value = {\n for name, machine in var.machines :\n \"${var.prefix}-${name}\" => machine.ip\n i"
},
{
"path": "contrib/terraform/vsphere/modules/kubernetes-cluster/templates/cloud-init.tpl",
"chars": 115,
"preview": "#cloud-config\n\nssh_authorized_keys:\n%{ for ssh_public_key in ssh_public_keys ~}\n - ${ssh_public_key}\n%{ endfor ~}\n"
},
{
"path": "contrib/terraform/vsphere/modules/kubernetes-cluster/templates/metadata.tpl",
"chars": 288,
"preview": "instance-id: ${hostname}\nlocal-hostname: ${hostname}\nnetwork:\n version: 2\n ethernets:\n ${interface_name}:\n mat"
},
{
"path": "contrib/terraform/vsphere/modules/kubernetes-cluster/templates/vapp-cloud-init.tpl",
"chars": 536,
"preview": "#cloud-config\n\nssh_authorized_keys:\n%{ for ssh_public_key in ssh_public_keys ~}\n - ${ssh_public_key}\n%{ endfor ~}\n\nwrit"
},
{
"path": "contrib/terraform/vsphere/modules/kubernetes-cluster/variables.tf",
"chars": 869,
"preview": "## Global ##\nvariable \"prefix\" {}\n\nvariable \"machines\" {\n description = \"Cluster machines\"\n type = map(object({\n no"
},
{
"path": "contrib/terraform/vsphere/modules/kubernetes-cluster/versions.tf",
"chars": 158,
"preview": "terraform {\n required_providers {\n vsphere = {\n source = \"hashicorp/vsphere\"\n version = \">= 1.24.3\"\n }"
},
{
"path": "contrib/terraform/vsphere/output.tf",
"chars": 508,
"preview": "output \"master_ip_addresses\" {\n value = module.kubernetes.master_ip\n}\n\noutput \"worker_ip_addresses\" {\n value = module."
},
{
"path": "contrib/terraform/vsphere/sample-inventory/cluster.tfvars",
"chars": 930,
"preview": "prefix = \"default\"\n\ninventory_file = \"inventory.ini\"\n\nmachines = {\n \"master-0\" : {\n \"node_type\" : \"master\",\n \"ip\""
},
{
"path": "contrib/terraform/vsphere/templates/inventory.tpl",
"chars": 206,
"preview": "\n[all]\n${connection_strings_master}\n${connection_strings_worker}\n\n[kube_control_plane]\n${list_master}\n\n[etcd]\n${list_mas"
},
{
"path": "contrib/terraform/vsphere/variables.tf",
"chars": 1413,
"preview": "## Global ##\n\n# Required variables\n\nvariable \"machines\" {\n description = \"Cluster machines\"\n type = map(object({\n n"
},
{
"path": "contrib/terraform/vsphere/versions.tf",
"chars": 158,
"preview": "terraform {\n required_providers {\n vsphere = {\n source = \"hashicorp/vsphere\"\n version = \">= 1.24.3\"\n }"
},
{
"path": "docs/CNI/calico.md",
"chars": 16468,
"preview": "# Calico\n\nCheck if the calico-node container is running\n\n```ShellSession\ndocker ps | grep calico\n```\n\nThe **calicoctl.sh"
},
{
"path": "docs/CNI/cilium.md",
"chars": 13368,
"preview": "# Cilium\n\n## Unprivileged agent configuration\n\nBy default, Cilium is installed with `securityContext.privileged: false`."
},
{
"path": "docs/CNI/cni.md",
"chars": 501,
"preview": "CNI\n==============\n\nThis network plugin only unpacks CNI plugins version `cni_version` into `/opt/cni/bin` and instructs"
},
{
"path": "docs/CNI/flannel.md",
"chars": 1874,
"preview": "# Flannel\n\nFlannel is a network fabric for containers, designed for Kubernetes\n\nSupported [backends](https://github.com/"
},
{
"path": "docs/CNI/kube-ovn.md",
"chars": 1795,
"preview": "# Kube-OVN\n\nKube-OVN integrates the OVN-based Network Virtualization with Kubernetes. It offers an advanced Container Ne"
},
{
"path": "docs/CNI/kube-router.md",
"chars": 3561,
"preview": "# Kube-router\n\nKube-router is a L3 CNI provider, as such it will setup IPv4 routing between\nnodes to provide Pods' netwo"
},
{
"path": "docs/CNI/macvlan.md",
"chars": 698,
"preview": "# Macvlan\n\n## How to use it\n\n* Enable macvlan in `group_vars/k8s_cluster/k8s_cluster.yml`\n\n```yml\n...\nkube_network_plugi"
}
]
// ... and 993 more files (download for full content)
About this extraction
This page contains the full source code of the kubernetes-sigs/kubespray GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 1193 files (3.1 MB), approximately 895.6k tokens, and a symbol index with 52 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.