The same 2 tier application is deployed in the beta namespace. It must display a green web page on success. Click on the App tab at the top of your terminal to view your application. It is currently failed. Troubleshoot and fix the issue.
Stick to the given architecture. Use the same names and port numbers as given in the below architecture diagram. Feel free to edit, delete or recreate objects as necessary.
1. Click on the `App` button at the top of the terminal. Observe the error message.
It is telling us
```
Can't connect to MySQL server on 'mysql-service:3306'
```
Which suggests that the service exists but the port settings may be incorrect.
1. Edit the service
```
kubectl edit service mysql-service -n beta
```
```yaml
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2023-10-14T11:54:04Z"
name: mysql-service
namespace: beta
resourceVersion: "1166"
uid: d6b07c71-5c49-4118-849a-0b12dd382597
spec:
clusterIP: 10.43.42.85
clusterIPs:
- 10.43.42.85
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- port: 3306 # <- Correct
protocol: TCP
targetPort: 8080 # <- Incorrect, should also be 3306
selector:
name: mysql
sessionAffinity: None
type: ClusterIP
```
The mysql database server listens on port `3306` meaning that is the port that its pod will expose, and where `targetPort` should be pointing. We can verify that like this
```
kubectl get po -n beta mysql -o jsonpath='{.spec.containers[*].ports[*].containerPort}'
```
or simply get the yaml for the pod.
3.
The same 2 tier application is deployed in the gamma namespace. It must display a green web page on success. Click on the App tab at the top of your terminal to view your application. It is currently failed or unresponsive. Troubleshoot and fix the issue.
Stick to the given architecture. Use the same names and port numbers as given in the below architecture diagram. Feel free to edit, delete or recreate objects as necessary.
1. Click on the `App` button at the top of the terminal. Observe the error message.
It is telling us
```
Can't connect to MySQL server on 'mysql-service:3306' (111 Connection refused)
```
Which suggests that the service exists but there is something wrong with it.
1. Edit the service
```
kubectl edit service mysql-service -n gamma
```
This time the name is correct, as are both the ports. So perhaps the pod selector is the issue
Open an additional terminal (`+` button above terminal) so as not to have to quit vi now, and run
```
kubectl get pods -n gamma --show-labels
```
Note the labels. Always ignore `pod-template-hash` label. It is used internally by kubernetes.
```
name=webapp-mysql,pod-template-hash=5456999f7b
```
That doesn't match with `sql00001` in the service selector. Switch back to your vi session in Terminal 1 and fix the selector to use the correct value for the `name` label.
```yaml
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2023-10-14T12:04:31Z"
name: mysql-service
namespace: gamma
resourceVersion: "1441"
uid: 872dfbcd-0b1d-4292-85f6-803d62d05b0a
spec:
clusterIP: 10.43.21.189
clusterIPs:
- 10.43.21.189
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- port: 3306
protocol: TCP
targetPort: 3306
selector:
name: sql00001 # <- Fix this
sessionAffinity: None
type: ClusterIP
```
4.
The same 2 tier application is deployed in the delta namespace. It must display a green web page on success. Click on the App tab at the top of your terminal to view your application. It is currently failed. Troubleshoot and fix the issue.
Stick to the given architecture. Use the same names and port numbers as given in the below architecture diagram. Feel free to edit, delete or recreate objects as necessary.
1. Click on the `App` button at the top of the terminal. Observe the error message.
It is telling us
```
Access denied for user 'sql-user'@'10.42.0.16'
```
So this means the application is using an incorrect mysql user account. The architecture diagram in the question tells you what the correct credentials are.
So this time the fix is going to be in the application deployment, rather than in the service.
1. Fix the deployment
```
kubectl edit deployment webapp-mysql -n delta
```
Scroll down to the container's environment section and fix the user name
```yaml
template:
metadata:
creationTimestamp: null
labels:
name: webapp-mysql
name: webapp-mysql
spec:
containers:
- env:
- name: DB_Host
value: mysql-service
- name: DB_User
value: sql-user # <- Fix this
- name: DB_Password
value: paswrd
image: mmumshad/simple-webapp-mysql
imagePullPolicy: Always
name: webapp-mysql
```
5.
The same 2 tier application is deployed in the epsilon namespace. It must display a green web page on success. Click on the App tab at the top of your terminal to view your application. It is currently failed. Troubleshoot and fix the issue.
The question indicates there are *two* issues that need fixing.
1. Click on the `App` button at the top of the terminal. Observe the error message.
It is telling us
```
Access denied for user 'sql-user'@'10.42.0.16'
```
So it is another authentication issue.
1. Check the deployment
Vaildate the environment against the values provided in the architecture diagram. Looks like the same issue as previous question, so fix that the same way.
1. Wait! We fixed that, but it's *still* not working!
Check the mysql pod, since mysql also needs some credential information on its end.
We cannot use `kubectl edit` to change values of a standalone POD.
```
kubectl get pod -n epsilon mysql -o yaml > mysql.yaml
vi mysql.yaml
```
```yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2023-10-14T12:21:51Z"
labels:
name: mysql
name: mysql
namespace: epsilon
resourceVersion: "2002"
uid: f369770b-be24-43c1-b754-d7e1396d6952
spec:
containers:
- env:
- name: MYSQL_ROOT_PASSWORD
value: passwooooorrddd # <- Fix this
image: mysql:5.6
imagePullPolicy: IfNotPresent
name: mysql
ports:
```
1. Recreate the pod
```
kubectl replace --force -f mysql.yaml
```
6.
The same 2 tier application is deployed in the zeta namespace. It must display a green web page on success. Click on the App tab at the top of your terminal to view your application. It is currently failed. Troubleshoot and fix the issue.
The question indicates there are *three* issues to fix.
1. Click on the `App` button at the top of the terminal. Observe the error message.
We get a `502 Bad Gateway` error.
This is indicative that the lab display infrastructure cannot connect to the service it's supposed to. Examine the URL in the browser address bar
```
30081-port-1795f98fde814933.labs.kodekloud.com/
```
On KodeKloud labs, the `30081-port` part indicates a node port it's trying to connect to. Note also that the infrastructure diagram states that `30081` should be the `web-service` nodeport.
1. Examine `web-service` since that's how we view the app.
```
kubectl edit service -n zeta web-service
```
```yaml
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2023-10-14T13:27:26Z"
name: web-service
namespace: zeta
resourceVersion: "1530"
uid: 9b655dda-675a-43ae-80e4-deadb3a38179
spec:
clusterIP: 10.43.48.98
clusterIPs:
- 10.43.48.98
externalTrafficPolicy: Cluster
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- nodePort: 30088 # <- Edit this
port: 8080
protocol: TCP
targetPort: 8080
selector:
name: webapp-mysql
sessionAffinity: None
type: NodePort
```
1. Retry the app in the browser
We have seen this message before! Fix it as per above, using namesapce `zeta`.
1. Retry the app in the browser
We have also seen this message before! Fix it as per above, using namesapce `zeta`.
================================================
FILE: docs/12-Troubleshooting/04-Control-Plane-Failure.md
================================================
# Control Plane Failure
- Take me to [Lecture](https://kodekloud.com/topic/control-plane-failure/)
- In this lecture we will use how to troubleshoot the Control Plane components.
- To check the status of the nodes if they are healthy
```
kubectl get nodes
```
- To check the status of the pods if the are running
```
kubectl get pods
```
- To check the status of all the pods of the Control Plane components(if they are deployed with kubeadm tool) and make sure they are **`Running`**
```
kubectl get pods -n kube-system
```
- If the Control Plane components are deployed as services then check the status of all the components
- To check the status of **`kube-apiserver`**
```
service kube-apiserver status
```
- To check the status of **`kube-controller-manager`**
```
service kube-controller-manager status
```
- To check the status of **`kube-scheduler`**
```
service kube-scheduler status
```
- To check the status of **`kubelet`**
```
service kubelet status
```
- To check the status of **`kube-proxy`** on the worker nodes.
```
service kube-proxy status
```
- To check the logs of the Control Plane components deployed as Pods:
```
kubectl logs kube-apiserver-master -n kube-system
```
- To check the logs of the Control Plane components deployed as SystemD Service
```
sudo journalctl -u kube-apiserver
```
================================================
FILE: docs/12-Troubleshooting/05-Practice-Test-Control-Plane-Failure.md
================================================
# Practice Test - Control Plane Failure
- Lets Debug the Failure of [Control Plane](https://kodekloud.com/topic/practice-test-control-plane-failure/)
================================================
FILE: docs/12-Troubleshooting/06-Solution-Control-Plane-Failure.md
================================================
# Solution Control Plane Failure
- Lets have a look at the [Practice Test](https://kodekloud.com/topic/practice-test-control-plane-failure/) of the Control Plane Failure
### Solution
1. Check Solution
```
kubectl get pods -n kube-system
```
```
sed -i 's/kube-schedulerrrr/kube-scheduler/g' /etc/kubernetes/manifests/kube-scheduler.yaml
```
2. Check Solution
```
kubectl scale deploy app --replicas=2
```
3. Check Solution
```
sed -i 's/controller-manager-XXXX.conf/controller-manager.conf/' /etc/kubernetes/manifests/kube-controller-manager.yaml
```
4. Check Solution
```
sed -i 's/WRONG-PKI-DIRECTORY/pki/' /etc/kubernetes/manifests/kube-controller-manager.yaml
```
================================================
FILE: docs/12-Troubleshooting/07-Worker-Node-Failure.md
================================================
# Worker Node Failure
- Take me to the [Lecture](https://kodekloud.com/topic/worker-node-failure/)
- Lets check the status of the Nodes in the cluster, are they **`Ready`** or **`NotReady`**
```
kubectl get nodes
```
- If they are **`NotReady`** then check the **`LastHeartbeatTime`** of the node to find out the time when node might have crashed
```
kubectl describe node worker-1
```
- Check the possible **`CPU`** and **`MEMORY`** using **`top`** and **`df -h`**
- Check the status and the logs of the **`kubelet`** for the possible issues.
```
serivce kubelet status
```
```
sudo journalctl -u kubelet
```
- Check the **`kubelet`** Certificates, they are not expired, and in the right group and issued by the right CA.
```
openssl x509 -in /var/lib/kubelet/worker-1.crt -text
```
================================================
FILE: docs/12-Troubleshooting/08-Practice-Test-Worker-Node-Failure.md
================================================
# Practice Test - Worker Node Failure
- Lets Debug the Failure of [Worker Node](https://kodekloud.com/topic/practice-test-worker-node-failure/)
================================================
FILE: docs/12-Troubleshooting/09-Solution-Worker-Node-Failure.md
================================================
# Solution Worker Node Failure
- Lets have a look at the [Practice Test](https://kodekloud.com/topic/practice-test-worker-node-failure/) of the Worker Node Failure
### Solution
1.
Fix the broken cluster
* Fix `node01`
1. Check the nodes
```bash
kubectl get nodes
```
We see that `node01` has a status of `NotReady`. This usually means that communication with the node's kubelet has been lost.
1. Go to the node and investigate
```bash
ssh node01
```
1. Check kubelet status
```bash
systemctl status kubelet
```
We can see from the output that kublet is not running, in fact it has exited. Therefore we should try starting it.
1. Start kubelet
```bash
systemctl start kubelet
```
1. Now check it is OK.
```bash
systemctl status kubelet
```
Now we can see it is `active (running)`, which is good.
1. Return to controlplane
```bash
exit
```
1. Check nodes again
```bash
kubectl get nodes
```
It is good!
2.
The cluster is broken again. Investigate and fix the issue.
* Fix cluster
1. Check the nodes
```bash
kubectl get nodes
```
We see that `node01` has a status of `NotReady`. This usually means that communication with the node's kubelet has been lost.
1. Go to the node and investigate
```bash
ssh node01
```
1. Check kubelet status
```bash
systemctl status kubelet
```
We can see from the output that it is crashlooping `activating (auto-restart)`, therefore this is likey a configuration issue.
1. Check kubelet logs
```bash
journalctl -u kubelet
```
There is a lot of information, however the error we are interested in, which is the cause of all other errors is this one
```
"failed to construct kubelet dependencies: unable to load client CA file /etc/kubernetes/pki/WRONG-CA-FILE.crt: open /etc/kubernetes/pki/WRONG-CA-FILE.crt: no such file or directory"
```
If kubelet cannot load its certificates, then it cannot autheticate with API server. This is a fatal error, so kubelet exits.
1. Check the indicated directory for certificates
```bash
ls -l /etc/kubernetes/pki
```
We see it contains `ca.crt` which we will assume is the correct certificate, therefore we need to find the kubelet configuration file and correct the error there.
1. Locate kubelet's configuration file
kubelet is an operating system service, so its service unit file will give us that info
```bash
systemctl cat kubelet
```
Note this line
```
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
```
There is the config YAML file
1. Fix configuration
```bash
vi /var/lib/kubelet/config.yaml
```
```yaml
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
anonymous:
enabled: false
webhook:
cacheTTL: 0s
enabled: true
x509:
clientCAFile: /etc/kubernetes/pki/WRONG-CA-FILE.crt # <- Fix this
authorization:
mode: Webhook
```
Note that you can perform the same edit with a single `sed` command. This is quicker than editing in vi.
```bash
sed -i 's/WRONG-CA-FILE.crt/ca.crt/g' /var/lib/kubelet/config.yaml
```
1. Check status
Wait a few seconds, kubelet will be auto-restarted.
```bash
systemctl status kubelet
```
Now we can see it is `active (running)`, which is good. If it is not, then you made a mistake when editing the config file, probably broke the YAML syntax or did not edit the certificate filename correctly. Return to step `vii.` above and fix it.
1. Return to controlplane
```bash
exit
```
1. Check nodes again
```bash
kubectl get nodes
```
It is good!
3.
The cluster is broken again. Investigate and fix the issue.
* Fix cluster
1. Check the nodes
```bash
kubectl get nodes
```
We see that `node01` has a status of `NotReady`. This usually means that communication with the node's kubelet has been lost.
1. Go to the node and investigate
```bash
ssh node01
```
1. Check kubelet status
```bash
systemctl status kubelet
```
We can see it is `active (running)`, however the API server still thinks there is an issue. So we must again go to the kubelet logs.
1. Check kubelet logs
```bash
journalctl -u kubelet
```
There is a lot of information, however the error we are interested in, which is the cause of all other errors is this one
```
"Unable to register node with API server" err="Post \"https://controlplane:6553/api/v1/nodes\": dial tcp 192.10.46.12:6553: connect: connection refused" node="node01"
```
What do you know about the usual port for API server? It's not `6553`! kubelet uses a kubeconfig file to connect to API server just like kubectl does, so we need to locate and fix that.
1. Locate kubelet's kubeconfig file
kubelet is an operating system service, so its service unit file will give us that info
```bash
systemctl cat kubelet
```
Note this line
```
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
```
There are two kubeconfigs. The first one is used when a node is created and is joining the cluster. The second one is used for normal operation. It is therefore the second one we are interested in.
1. Fix the kubeconfig
Port should be `6443`
```bash
vi /etc/kubernetes/kubelet.conf
```
```yaml
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: REDACTED
server: https://controlplane:6553 # <- Fix this
name: default-cluster
contexts:
- context:
cluster: default-cluster
namespace: default
user: default-auth
name: default-context
current-context: default-context
kind: Config
preferences: {}
users:
- name: default-auth
user:
client-certificate: /var/lib/kubelet/pki/kubelet-client-current.pem
client-key: /var/lib/kubelet/pki/kubelet-client-current.pem
```
Note that you can perform the same edit with a single `sed` command. This is quicker than editing in vi.
```bash
sed -i 's/6553/6443/g' /etc/kubernetes/kubelet.conf
```
1. Restart kubelet
Since kubelet is already running (not crashlooping), we need to restart it so it gets the updated kubeconfig
```bash
systemctl restart kubelet
```
1. Check status
```bash
systemctl status kubelet
```
Now we can see it is `active (running)`, which is good. If it is not, then you made a mistake when editing the kubeconfig, probably broke the YAML syntax. Return to step `vi.` above and fix it.
1. Return to controlplane
```bash
exit
```
1. Check nodes again
```bash
kubectl get nodes
```
It is good! If it is not, then you probably made a mistake setting the port number. Return to `node01` and redo from step `vi.` above.
================================================
FILE: docs/12-Troubleshooting/10-Practice-Test-Troubleshoot-Network.md
================================================
# Solution Troubleshoot Network
Lets have a look at the [Practice Test](https://kodekloud.com/topic/practice-test-troubleshoot-network/) of the Troubleshoot Network
Note that this lab is sequential. You must solve test 1 completely before you can solve test 2, i.e. you cannot skip test 1 and do test 2 only.
1.
Troubleshooting Test 1
We are asked to ensure all the components are working, so first let's examine the cluster to see what state it is in.
How many nodes, and their status?
```
kubectl get nodes
```
Seems OK...
Next, the pods
```
kubectl get pods -A
```
Now we see that the `webapp` and `mysql` pods are stuck at `ContainerCreating`. We need to describe the pods and check the errors.
You will note that they are complaining about `plugin type="weave-net" name="weave" failed (add): unable to allocate IP address`, so clearly we have a networking issue and it's related to Weave.
When you did the `get pods` above, did you see any evidence of network support containers, like `weave`?
No - so we need to install networking support.
Let's install `Weave`
```
kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml
```
Now wait for a minute or so for it to initialize, then check the application pods
```
kubectl get pods -n triton
```
1.
Troubleshooting Test 2
Once again let's examine the cluster to see what state it is in.
How many nodes, and their status?
```
kubectl get nodes
```
Seems OK...
Next, the pods
```
kubectl get pods -A
```
The kube-proxy pod is not running. It is actually crash-looping which means it tries to start, then fails. As a result the rules needed to allow connectivity to the services have not been created. First place to look when diagnosing CrashLoopBackoff is the pod logs.
1. Check the logs of the kube-proxy pod
```
kubectl -n kube-system logs
```
We see that it cannot find a configuration file.
Now try looking for the configuration in case it has a different name
```
ls -l /var/lib/kube-proxy
```
The directory is not found!
1. Inspect the pod template spec in the `kube-proxy` daemonset.
```
kubectl get ds -n kube-system kube-proxy -o yaml | less
```
Scroll around and check volumes and volume mounts. Notice that a config map is mounted at the path `/var/lib/kube-proxy` within the pod.
1. Inspect the config map
```
kubectl describe cm -n kube-system kube-proxy
```
Here we see that the files mounted by the config map are `config.conf` and `kubeconfig.conf`, but _not_ `configuration.conf`.
These two files are
* `config.conf` - This is the actual configuration that kube-proxy needs to load. This file refers to `kubeconfig.conf`
* `kubeconfig.conf` - This is simply a kubeconfig file, same as you will find on the lab terminal in `~/.kube/config`. It is the credentials and address for kube-proxy to talk to the api server.
1. Fix the command line arguments to `kube-proxy`
```
kubectl edit ds -n kube-system kube-proxy
```
Set the correct filename
```
--config=/var/lib/kube-proxy/config.conf
```
Finally, confirm it is running.
```
kubectl get pods -n kube-system
```
================================================
FILE: docs/13-Other-Topics/01-Labs-JSON-PATH.md
================================================
# Labs JSON PATH
- I want to learn JSON, Take me to [JSON PATH](https://kodekloud.com/topic/labs-json-path-3/)
================================================
FILE: docs/13-Other-Topics/02-Pre-Requisites-JSON-PATH.md
================================================
# Pre-requisite for JSON PATH
- I want to know more about JSON PATH and its [Pre-requisite](https://kodekloud.com/topic/pre-requisites-json-path/)
================================================
FILE: docs/13-Other-Topics/03-Advance-Kubectl-Commands.md
================================================
# Advance Kubectl Commands
- Take me to the [Lecture](https://kodekloud.com/topic/advanced-kubectl-commands/)
- To get the output of **`kubectl`** in a json format:
```
kubectl get nodes -o json
```
```
kubectl get pods -o json
```
- To get the image name used by pod via json path query:
```
kubectl get pods -o=jsonpath='{.items[0].spec.containers[0].image}'
```
- To get the names of node in the cluster:
```
kubectl get pods -o=jsonpath='{.items[*].metadata.name}'
```
- To get the architecture of node in the cluster:
```
kubectl get pods -o=jsonpath='{.items[*].status.nodeInfo.architecture}'
```
- To get the count of the cpu of node in the cluster:
```
kubectl get pods -o=jsonpath='{.items[*].status.status.capacity.cpu}'
```
#### Loops - Range
- To print the output in a separate column (one column with node name and other with CPU count):
```
kubectl get nodes -o=custom-columns=NODE:.metadata.name ,CPU:.status.capacity.cpu
```
- Kubectl comes with a **`sort by`** property which can be combined with json path query to **`sort`** by name or **`CPU count`**
```
kubectl get nodes --sort-by=.metadata.name
```
```
kubectl get nodes --sort-by=..status.capacity.cpu
```
================================================
FILE: docs/13-Other-Topics/04-Practice-Test-Advance-Kubectl-Commands.md
================================================
# Practice Test for Advance Kubectl Commands
- Take me to [Advance Practice Test for Kubectl Commands](https://kodekloud.com/topic/practice-test-advanced-kubectl-commands/)
### Solution
1. Check Solution
```
kubectl get nodes -o json > /opt/outputs/nodes.json
```
2. Check Solution
```
kubectl get node node01 -o json > /opt/outputs/node01.json
```
3. Check Solution
```
kubectl get nodes -o=jsonpath='{.items[*].metadata.name}' > /opt/outputs/node_names.txt
```
4. Check Solution
```
kubectl get nodes -o jsonpath='{.items[*].status.nodeInfo.osImage}' > /opt/outputs/nodes_os.txt
```
5. Check Solution
```
kubectl config view --kubeconfig=my-kube-config -o jsonpath="{.users[*].name}" > /opt/outputs/users.txt
```
6. Check Solution
```
kubectl get pv --sort-by=.spec.capacity.storage > /opt/outputs/storage-capacity-sorted.txt
```
7. Check Solution
```
kubectl get pv --sort-by=.spec.capacity.storage -o=custom-columns=NAME:.metadata.name,CAPACITY:.spec.capacity.storage > /opt/outputs/pv-and-capacity-sorted.txt
```
8. Check Solution
```
kubectl config view --kubeconfig=my-kube-config -o jsonpath="{.contexts[?(@.context.user=='aws-user')].name}" > /opt/outputs/aws-context-name
```
================================================
FILE: docs/14-Lightning-Labs/01-Lightning-Labs-Introduction.md
================================================
# Lightning Labs Introduction
#### Welcome to the KodeKloud CKA Lightning Labs!
- This section has been created to give you hands-on practice in solving questions of mixed difficulty in a short period of time.
- This environment is only valid for 35 minutes. You have 5-8 questions to complete within this time.
- You can toggle between the questions but make sure that that you click on `END EXAM` before the 35 minute mark. To pass, you need to secure 80%.
Good Luck!!!
Note: Answers for most questions should be available under the `/var/answers` directory on the master node.
#### Disclaimer:
- Please note that this exam is not a replica of the actual exam
- Please note that the questions in these exams are not the same as in the actual exam
- Please note that the interface is not the same as in the actual exam
- Please note that the scoring system may not be the same as in the actual exam
- Please note that the difficulty level may not be the same as in the actual exam
- I want to understand what Lightning Lab is, [Let's Explore](https://kodekloud.com/topic/lightning-lab-introduction/)
================================================
FILE: docs/14-Lightning-Labs/02-Lightning-Lab-1.md
================================================
# Lightning Lab 1
- I am ready! [Take me to Lightning Lab 1](https://kodekloud.com/topic/lightning-lab-1-2/)
## Solution to LL-1
1.
Upgrade the current version of kubernetes from 1.28.0 to 1.29.0 exactly using the kubeadm utility.
Make sure that the upgrade is carried out one node at a time starting with the controlplane node. To minimize downtime, the deployment `gold-nginx` should be rescheduled on an alternate node before upgrading each node.
Upgrade `controlplane` node first and drain node `node01` before upgrading it. Pods for `gold-nginx` should run on the controlplane node subsequently.
**Upgrade `controlplane`**
1. Update package repo
```bash
apt update
```
1. Check madison to see what kubernetes packages are available
```bash
apt-cache madison kubeadm
```
Note that only 1.28 versions are present, meaning you have to grab the 1.29 repos
1. Grab kubernetes 1.29 repos
For this, we need to edit the apt source file which you should find is `/etc/apt/sources.list.d/kubernetes.list`
```bash
vi /etc/apt/sources.list.d/kubernetes.list
```
FInd any occurrence of `1.28` in this file and change it to `1.29`, then save and exit from vi.
1. Now run madison again to find out the package version for 1.29
```bash
apt-cache madison kubeadm
```
You should see the following in the list
> `kubeadm | 1.29.0-1.1 | https://pkgs.k8s.io/core:/stable:/v1.29/deb Packages`
Now we know the package version is `1.29.0-1.1` we can proceed with the upgrade
1. Drain node
```
kubectl drain controlplane --ignore-daemonsets
```
1. Upgrade kubeadm
```
apt-mark unhold kubeadm
apt install -y kubeadm=1.29.0-1.1
```
1. Plan and apply upgrade
```
kubeadm upgrade plan
kubeadm upgrade apply v1.29.0
```
1. Upgrade the kubelet
```
apt-mark unhold kubelet
apt install -y kubelet=1.29.0-1.1
systemctl daemon-reload
systemctl restart kubelet
```
1. Reinstate controlplane node
```
kubectl uncordon controlplane
```
1. Upgrade kubectl
```
apt-mark unhold kubectl
apt install -y kubectl=1.29.0-1.1
```
1. Re-hold packages
```
apt-mark hold kubeadm kubelet kubectl
```
**Upgrade `node01`**
1. Drain the worker node
```
kubectl drain node01 --ignore-daemonsets
```
1. Go to worker node
```
ssh node01
```
1. As before, you will need to update the package caches for v1.29
Follow the same steps as you did on `controlplane`
1. Upgrade kubeadm
```
apt-mark unhold kubeadm
apt install -y kubeadm=1.29.0-1.1
```
1. Upgrade node
```
kubeadm upgrade node
```
1. Upgrade the kubelet
```
apt-mark unhold kubelet
apt install kubelet=1.29.0-1.1
systemctl daemon-reload
systemctl restart kubelet
```
1. Re-hold packages
```
apt-mark hold kubeadm kubelet
```
1. Return to controlplane
```
exit
```
1. Reinstate worker node
```
kubectl uncordon node01
```
1. Verify `gold-nginx` is scheduled on controlplane node
```
kubectl get pods -o wide | grep gold-nginx
```
**TIP**
To do cluster upgrades faster and save at least 3 minutes, you can work on both nodes at the same time.
While `kubeadm upgrade apply` is running on `controlplane`, which takes some minutes, open a second terminal and perform steps `ii`, `iii` and `iv` of "Upgrade `node01`", so that it is ready for `kubeadm upgrade node` as soon as you have drained it.
2.
Print the names of all deployments in the admin2406 namespace in the following format...
This is a job for `custom-columns` output of kubectl
```
kubectl -n admin2406 get deployment -o custom-columns=DEPLOYMENT:.metadata.name,CONTAINER_IMAGE:.spec.template.spec.containers[].image,READY_REPLICAS:.status.readyReplicas,NAMESPACE:.metadata.namespace --sort-by=.metadata.name > /opt/admin2406_data
```
3.
A kubeconfig file called admin.kubeconfig has been created in /root/CKA. There is something wrong with the configuration. Troubleshoot and fix it.
First, let's test this kubeconfig
```
kubectl get pods --kubeconfig /root/CKA/admin.kubeconfig
```
Notice the error message.
Now look at the default kubeconfig for the correct setting.
```
cat ~/.kube/config
```
Make the correction
```
vi /root/CKA/admin.kubeconfig
```
Test
```
kubectl get pods --kubeconfig /root/CKA/admin.kubeconfig
```
4.
Create a new deployment called nginx-deploy, with image nginx:1.16 and 1 replica. Next upgrade the deployment to version 1.17 using rolling update.
```
kubectl create deployment nginx-deploy --image=nginx:1.16
kubectl set image deployment/nginx-deploy nginx=nginx:1.17 --record
```
You may ignore the deprecation warning.
5.
A new deployment called alpha-mysql has been deployed in the alpha namespace. However, the pods are not running. Troubleshoot and fix the issue.
The deployment should make use of the persistent volume alpha-pv to be mounted at /var/lib/mysql and should use the environment variable MYSQL_ALLOW_EMPTY_PASSWORD=1 to make use of an empty root password.
Important: Do not alter the persistent volume.
Inspect the deployment to check the environment variable is set. Here I'm using `yq` which is like `jq` but for YAML to not have to view the _entire_ deployment YAML, just the section beneath `containers` in the deployment spec.
```
kubectl get deployment -n alpha alpha-mysql -o yaml | yq e .spec.template.spec.containers -
```
Find out why the deployment does not have minimum availability. We'll have to find out the name of the deployment's pod first, then describe the pod to see the error.
```
kubectl get pods -n alpha
kubectl describe pod -n alpha alpha-mysql-xxxxxxxx-xxxxx
```
We find that the requested PVC isn't present, so create it. First, examine the Persistent Volume to find the values for access modes, capacity (storage), and storage class name
```
kubectl get pv alpha-pv
```
Now use `vi` to create a PVC manifest
```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql-alpha-pvc
namespace: alpha
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: slow
```
6.
Take the backup of ETCD at the location /opt/etcd-backup.db on the controlplane node.
This question is a bit poorly worded. It requires us to make a backup of etcd and store the backup at the given location.
Know that the certificates we need for authentication of `etcdctl` are located in `/etc/kubernetes/pki/etcd`
```
ETCDCTL_API='3' etcdctl snapshot save \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key \
/opt/etcd-backup.db
```
Whilst we _could_ also use the argument `--endpoints=127.0.0.1:2379`, it is not necessary here as we are on the controlplane node, same as `etcd` itself. The default endpoint is the local host.
7.
Create a pod called secret-1401 in the admin1401 namespace using the busybox image....
The container within the pod should be called `secret-admin` and should sleep for 4800 seconds.
The container should mount a read-only secret volume called secret-volume at the path `/etc/secret-volume`. The secret being mounted has already been created for you and is called `dotfile-secret`.
1. Use imperative command to get a starter manifest
```
kubectl run secret-1401 -n admin1401 --image busybox --dry-run=client -o yaml --command -- sleep 4800 > admin.yaml
```
1. Edit this manifest to add in the details for mounting the secret
```
vi admin.yaml
```
Add in the volume and volume mount sections seen below
```yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: secret-1401
name: secret-1401
namespace: admin1401
spec:
volumes:
- name: secret-volume
secret:
secretName: dotfile-secret
containers:
- command:
- sleep
- "4800"
image: busybox
name: secret-admin
volumeMounts:
- name: secret-volume
readOnly: true
mountPath: /etc/secret-volume
```
1. And create the pod
```
kubectl create -f admin.yaml
```
================================================
FILE: docs/15-Mock-Exams/01-Introduction.md
================================================
# Mock Exams Introduction
- Take me to [Introduction of Mock Exams](https://kodekloud.com/topic/mock-exam-introduction-4/)
================================================
FILE: docs/15-Mock-Exams/02-Mock-Exam-1.md
================================================
# Mock Exam 1
Test My Knowledge, Take me to [Mock Exam 1](https://kodekloud.com/topic/mock-exam-1-3/)
#### Solution to the Mock Exam 1
1. Apply below manifests:
```
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: nginx-pod
name: nginx-pod
spec:
containers:
- image: nginx:alpine
name: nginx-pod
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
```
2. Run below command which create a pod with labels:
```
kubectl run messaging --image=redis:alpine --labels=tier=msg
```
3. Run below command to create a namespace:
```
kubectl create namespace apx-x9984574
```
4. Use the below command which will redirect the o/p:
```
kubectl get nodes -o json > /opt/outputs/nodes-z3444kd9.json
```
5. Execute below command which will expose the pod on port 6379:
```
kubectl expose pod messaging --port=6379 --name messaging-service
```
6. Apply below manifests:
```
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: hr-web-app
name: hr-web-app
spec:
replicas: 2
selector:
matchLabels:
app: hr-web-app
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: hr-web-app
spec:
containers:
- image: kodekloud/webapp-color
name: webapp-color
resources: {}
status: {}
```
In v1.19, we can add `--replicas` flag with `kubectl create deployment` command:
```
kubectl create deployment hr-web-app --image=kodekloud/webapp-color --replicas=2
```
7. To Create a static pod, copy it to the static pods directory. In this case, it is `/etc/kubernetes/manifests`. Apply below manifests:
```
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: static-busybox
name: static-busybox
spec:
containers:
- command:
- sleep
- "1000"
image: busybox
name: static-busybox
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
```
8. Run below command to create a pod in namespace `finance`:
```
kubectl run temp-bus --image=redis:alpine -n finance
```
9. Run below command and troubleshoot step by step:
```
kubectl describe pod orange
```
Export the running pod using below command and correct the spelling of the command **`sleeeep`** to **`sleep`**
```
kubectl get pod orange -o yaml > orange.yaml
```
Delete the running Orange pod and recreate the pod using command.
```
kubectl delete pod orange
kubectl create -f orange.yaml
```
10. Apply below manifests:
```
apiVersion: v1
kind: Service
metadata:
creationTimestamp: null
labels:
app: hr-web-app
name: hr-web-app-service
spec:
ports:
- port: 8080
protocol: TCP
targetPort: 8080
nodePort: 30082
selector:
app: hr-web-app
type: NodePort
status:
loadBalancer: {}
```
11. Run the below command to redirect the o/p:
```
kubectl get nodes -o jsonpath='{.items[*].status.nodeInfo.osImage}' > /opt/outputs/nodes_os_x43kj56.txt
```
12. Apply the below manifest to create a PV:
```
apiVersion: v1
kind: PersistentVolume
metadata:
name: pv-analytics
spec:
capacity:
storage: 100Mi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
hostPath:
path: /pv/data-analytics
```
================================================
FILE: docs/15-Mock-Exams/03-Mock-Exam-2.md
================================================
# Mock Exam 2
Level Up!
Take me to [Mock Exam 2](https://kodekloud.com/topic/mock-exam-2-3/)
================================================
FILE: docs/15-Mock-Exams/04-CKA-MockExam-2-Solution.md
================================================
# Mock Exam 2 Solution
1. Run the below command for solution:
```
ETCDCTL_API=3 etcdctl snapshot save --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key --endpoints=127.0.0.1:2379 /opt/etcd-backup.db
```
2. Run the below command for solution:
```
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: redis-storage
name: redis-storage
spec:
volumes:
- name: redis-storage
emptyDir: {}
containers:
- image: redis:alpine
name: redis-storage
resources: {}
volumeMounts:
- name: redis-storage
mountPath: /data/redis
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
```
3. Run the below command for solution:
```
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
name: super-user-pod
spec:
containers:
- image: busybox:1.28
name: super-user-pod
command: ["sleep", "4800"]
securityContext:
capabilities:
add: ["SYS_TIME"]
```
4. Run the below command for solution:
```
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: my-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Mi
```
```
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: use-pv
name: use-pv
spec:
containers:
- image: nginx
name: use-pv
volumeMounts:
- mountPath: "/data"
name: mypod
volumes:
- name: mypod
persistentVolumeClaim:
claimName: my-pvc
```
5. Run the below command for solution:
For Kubernetes Version <=1.17
```
kubectl run nginx-deploy --image=nginx:1.16 --replicas=1 --record
kubectl rollout history deployment nginx-deploy
kubectl set image deployment/nginx-deploy nginx=nginx:1.17 --record
kubectl rollout history deployment nginx-deploy
```
For Kubernetes Version >1.17
```
kubectl create deployment nginx-deploy --image=nginx:1.16 --dry-run=client -o yaml > deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deploy
spec:
replicas: 1
selector:
matchLabels:
app: nginx-deploy
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: nginx-deploy
spec:
containers:
- image: nginx:1.16
name: nginx
```
```
kubectl create -f deploy.yaml --record
kubectl rollout history deployment nginx-deploy
kubectl set image deployment/nginx-deploy nginx=nginx:1.17 --record
kubectl rollout history deployment nginx-deploy
```
6. Run the below command for solution:
```
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
name: john-developer
spec:
signerName: kubernetes.io/kube-apiserver-client
request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ1ZEQ0NBVHdDQVFBd0R6RU5NQXNHQTFVRUF3d0VhbTlvYmpDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRApnZ0VQQURDQ0FRb0NnZ0VCQUt2Um1tQ0h2ZjBrTHNldlF3aWVKSzcrVVdRck04ZGtkdzkyYUJTdG1uUVNhMGFPCjV3c3cwbVZyNkNjcEJFRmVreHk5NUVydkgyTHhqQTNiSHVsTVVub2ZkUU9rbjYra1NNY2o3TzdWYlBld2k2OEIKa3JoM2prRFNuZGFvV1NPWXBKOFg1WUZ5c2ZvNUpxby82YU92czFGcEc3bm5SMG1JYWpySTlNVVFEdTVncGw4bgpjakY0TG4vQ3NEb3o3QXNadEgwcVpwc0dXYVpURTBKOWNrQmswZWhiV2tMeDJUK3pEYzlmaDVIMjZsSE4zbHM4CktiSlRuSnY3WDFsNndCeTN5WUFUSXRNclpUR28wZ2c1QS9uREZ4SXdHcXNlMTdLZDRaa1k3RDJIZ3R4UytkMEMKMTNBeHNVdzQyWVZ6ZzhkYXJzVGRMZzcxQ2NaanRxdS9YSmlyQmxVQ0F3RUFBYUFBTUEwR0NTcUdTSWIzRFFFQgpDd1VBQTRJQkFRQ1VKTnNMelBKczB2czlGTTVpUzJ0akMyaVYvdXptcmwxTGNUTStsbXpSODNsS09uL0NoMTZlClNLNHplRlFtbGF0c0hCOGZBU2ZhQnRaOUJ2UnVlMUZnbHk1b2VuTk5LaW9FMnc3TUx1a0oyODBWRWFxUjN2SSsKNzRiNnduNkhYclJsYVhaM25VMTFQVTlsT3RBSGxQeDNYVWpCVk5QaGhlUlBmR3p3TTRselZuQW5mNm96bEtxSgpvT3RORStlZ2FYWDdvc3BvZmdWZWVqc25Yd0RjZ05pSFFTbDgzSkljUCtjOVBHMDJtNyt0NmpJU3VoRllTVjZtCmlqblNucHBKZWhFUGxPMkFNcmJzU0VpaFB1N294Wm9iZDFtdWF4bWtVa0NoSzZLeGV0RjVEdWhRMi80NEMvSDIKOWk1bnpMMlRST3RndGRJZjAveUF5N05COHlOY3FPR0QKLS0tLS1FTkQgQ0VSVElGSUNBVEUgUkVRVUVTVC0tLS0tCg==
usages:
- digital signature
- key encipherment
- client auth
groups:
- system:authenticated
```
```
kubectl certificate approve john-developer
kubectl create role developer --resource=pods --verb=create,list,get,update,delete --namespace=development
kubectl create rolebinding developer-role-binding --role=developer --user=john --namespace=development
kubectl auth can-i update pods --as=john --namespace=development
```
7. Run the below command for solution:
```
kubectl run nginx-resolver --image=nginx
kubectl expose pod nginx-resolver --name=nginx-resolver-service --port=80 --target-port=80 --type=ClusterIP
kubectl run test-nslookup --image=busybox:1.28 --rm -it --restart=Never -- nslookup nginx-resolver-service
kubectl run test-nslookup --image=busybox:1.28 --rm -it --restart=Never -- nslookup nginx-resolver-service > /root/CKA/nginx.svc
Get the IP of the nginx-resolver pod and replace the dots(.) with hyphon(-) which will be used below.
kubectl get pod nginx-resolver -o wide
kubectl run test-nslookup --image=busybox:1.28 --rm -it --restart=Never -- nslookup > /root/CKA/nginx.pod
```
8. Run the below command for solution:
```
kubectl run nginx-critical --image=nginx --dry-run=client -o yaml > static.yaml
cat static.yaml - Copy the contents of this file.
kubectl get nodes -o wide
ssh node01
OR
ssh
Check if static-pod directory is present which is /etc/kubernetes/manifests if not then create it.
mkdir -p /etc/kubernetes/manifests
Paste the contents of the file(static.yaml) copied in the first step to file nginx-critical.yaml.
Move/copy the nginx-critical.yaml to path /etc/kubernetes/manifests/
cp nginx-critical.yaml /etc/kubernetes/manifests
Go back to master node
kubectl get pods
```
================================================
FILE: docs/15-Mock-Exams/05-Mock-Exam-3.md
================================================
# Mock Exam 3
Wohooo! Level Up!
Take me to [Mock Exam 3](https://kodekloud.com/topic/mock-exam-3-2/)
================================================
FILE: docs/15-Mock-Exams/06-CKA-MockExam-3-Solution.md
================================================
# Mock Exam 3 Solution
1. Run the below command for solution:
```
kubectl create serviceaccount pvviewer
kubectl create clusterrole pvviewer-role --resource=persistentvolumes --verb=list
kubectl create clusterrolebinding pvviewer-role-binding --clusterrole=pvviewer-role --serviceaccount=default:pvviewer
```
```
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: pvviewer
name: pvviewer
spec:
containers:
- image: redis
name: pvviewer
resources: {}
serviceAccountName: pvviewer
```
2. Run the below command for solution:
```
kubectl get nodes -o jsonpath='{.items[*].status.addresses[?(@.type=="InternalIP")].address}' > /root/CKA/node_ips
```
3. Run the below command for solution:
```
apiVersion: v1
kind: Pod
metadata:
name: multi-pod
spec:
containers:
- image: nginx
name: alpha
env:
- name: name
value: alpha
- image: busybox
name: beta
command: ["sleep", "4800"]
env:
- name: name
value: beta
status: {}
```
4. Run the below command for solution:
```
apiVersion: v1
kind: Pod
metadata:
name: non-root-pod
spec:
securityContext:
runAsUser: 1000
fsGroup: 2000
containers:
- name: non-root-pod
image: redis:alpine
```
5. Run the below command for solution:
```
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: ingress-to-nptest
namespace: default
spec:
podSelector:
matchLabels:
run: np-test-1
policyTypes:
- Ingress
ingress:
- ports:
- protocol: TCP
port: 80
```
6. Run the below command for solution:
```
kubectl taint node node01 env_type=production:NoSchedule
```
Deploy `dev-redis` pod and to ensure that workloads are not scheduled to this `node01` worker node.
```
kubectl run dev-redis --image=redis:alpine
kubectl get pods -owide
```
Deploy new pod `prod-redis` with toleration to be scheduled on `node01` worker node.
```
apiVersion: v1
kind: Pod
metadata:
name: prod-redis
spec:
containers:
- name: prod-redis
image: redis:alpine
tolerations:
- effect: NoSchedule
key: env_type
operator: Equal
value: production
```
View the pods with short details:
```
kubectl get pods -owide | grep prod-redis
```
7. Run the below command for solution:
```
kubectl create namespace hr
kubectl run hr-pod --image=redis:alpine --namespace=hr --labels=environment=production,tier=frontend
```
8. Run the below command for solution:
```
vi /root/CKA/super.kubeconfig
Change the 2379 port to 6443 and run the below command to verify
kubectl cluster-info --kubeconfig=/root/CKA/super.kubeconfig
```
9. Run the below command for solution:
```
sed -i 's/kube-contro1ler-manager/kube-controller-manager/g' kube-controller-manager.yaml
```
================================================
FILE: docs/16-Ultimate-Mocks/02-Troubleshooting/README.md
================================================
# SECTION: Troubleshooting
* [19 Cluster 1, Network Policy, cyan-pod-cka28-trb](./docs/19-C1-netpol-cyan-pod-cka28-trb.md)
================================================
FILE: docs/16-Ultimate-Mocks/02-Troubleshooting/docs/11-C1-orange-pvc-cka13-trb.md
================================================
# Bind the orange-pvc-cka13-trb
For this question, please set the context to cluster1 by running:
```
kubectl config use-context cluster1
```
There is an existing persistent volume called orange-pv-cka13-trb. A persistent volume claim called orange-pvc-cka13-trb is created to claim storage from orange-pv-cka13-trb.
However, this PVC is stuck in a Pending state. As of now, there is no data in the volume.
Troubleshoot and fix this issue, making sure that orange-pvc-cka13-trb PVC is in Bound state.
---
### Solution
1. Describe the PVC and determine the issue
```
kubectl describe pvc orange-pvc-cka13-trb
```
Note the message "requested PV is too small". We must adjust the PVC to fit
2. Describe the PV and determine its properties. Note that PVC properties must be adjusted to match
```
kubectl describe pv orange-pv-cka13-trb
```
3. Adjust the PVC. Note that you cannot directly edit a PVC size to be smaller, so we have to replace it.
```
kubectl get pvc orange-pvc-cka13-trb -o yaml > pvc.yaml
vi pvc.yaml
```
Change the requested size to match the size of the PV. Save and exit vi, then replace the PVC with the edited manifest:
```
kubectl replace --force -f pvc.yaml
```
================================================
FILE: docs/16-Ultimate-Mocks/02-Troubleshooting/docs/19-C1-netpol-cyan-pod-cka28-trb.md
================================================
# Cluster 1, NetPol, cyan-pod-cka28-trb
For this question, please set the context to cluster1 by running:
```
kubectl config use-context cluster1
```
One of the nginx based pod called `cyan-pod-cka28-trb` is running under `cyan-ns-cka28-trb` namespace and it is exposed within the cluster using `cyan-svc-cka28-trb` service.
This is a restricted pod so a network policy called `cyan-np-cka28-trb` has been created in the same namespace to apply some restrictions on this pod.
Two other pods called `cyan-white-cka28-trb1` and `cyan-black-cka28-trb` are also running in the default namespace.
The nginx based app running on the `cyan-pod-cka28-trb` pod is exposed internally on the default nginx port (80).
**Expectation**: This app should only be accessible from the `cyan-white-cka28-trb` pod.
**Problem**: This app is not accessible from anywhere.
Troubleshoot this issue and fix the connectivity as per the requirement listed above.
Note: You can exec into `cyan-white-cka28-trb` and `cyan-black-cka28-trb` pods and test connectivity using the curl utility.
You may update the network policy, but make sure it is not deleted from the `cyan-ns-cka28-trb` namespace.
---
### Update - Intermittent lab bug!
The solution given below is correct, however in some instances it doesn't work due to an intermittent bug in the installation of Weave to the lab environment found by a very astute community member in [this thread](https://kodekloud.com/community/t/network-policy-blocking-all-the-ingress-traffic/300501/15?u=alistair_kodekloud) on the community forum.
TL;DR - To detect the presence of this bug, run the following two commands. Bonus - see if you can understand how they work! Note that the first one is split across multiple lines with `\` for legibility. This is a valid construct in shell script.
```
kubectl exec -n kube-system \
$(kubectl get po -n kube-system --selector name=weave-net -o jsonpath='{.items[0].metadata.name}') \
-c weave -- printenv | grep IPALLOC
kubectl get configmap -n kube-system kube-proxy -o jsonpath={'.data.config\.conf}' | yq e .clusterCIDR -
```
Both should report the same CIDR range, e.g. `10.244.0.0/16`. If they are not both the same (doesn't matter what they actually are, but must be the same), then the lab has the bug. Should you encounter this (netpol not working even though you have followed the solution below), then practice your skills of [manual pod scheduling](../../../03-Scheduling/02-Manual-Scheduling.md), and get all three concerned pods to restart on the same worker node (choose either node). Then the netpol should take effect.
### Solution
First, let's examine the policy we have
```
k get netpol -n cyan-ns-cka28-trb cyan-np-cka28-trb -o yaml
```
> Output. (The additional metadata is omitted as it is different every time)
```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: cyan-np-cka28-trb
namespace: cyan-ns-cka28-trb
spec:
egress:
- ports:
- port: 8080
protocol: TCP
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: default
ports:
- port: 8080
protocol: TCP
podSelector:
matchLabels:
app: cyan-app-cka28-trb
policyTypes:
- Ingress
- Egress
status: {}
```
The egress policy you find here is a [red herring](https://dictionary.cambridge.org/dictionary/english/red-herring). Since we are not concerned with egress from the nginx pod, only ingress to it from the other pods, then it does not feature in the solution to this problem so you can ignore it.
There are two issues that need fixing here. You can modify the policy and make both these changes with a single invocation of:
```
kubectl edit netpol -n cyan-ns-cka28-trb cyan-np-cka28-trb
```
1. The reason nothing can connect at the start is that the ingress port 8080 in the netpol is wrong. It should be 80. Why? We are told in the question that the nginx app in the pod to which the policy applies is listening on the default port `80`. Therefore the *ingress* port needs to be `80` and not `8080`. Fix this.
1. Now that’s fixed, everything in default namespace now has access to the pod on port 80, and curl will return the nginx default message. Thus we need to add to the rule a podSelector to ensure the incoming traffic can only come from the nominated pod in the default namespace, so it’s an AND rule.
The finished product is this. Again I have omitted the additional metadata but you can leave it in. Save and exit `vi` so the changes are applied.
```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: cyan-np-cka28-trb
namespace: cyan-ns-cka28-trb
spec:
egress:
- ports:
- port: 8080
protocol: TCP
ingress:
- from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: default
podSelector: # <- This was added. No dash before podSelector!
matchLabels:
app: cyan-white-cka28-trb
ports:
- port: 80 # <- This was edited
protocol: TCP
podSelector:
matchLabels:
app: cyan-app-cka28-trb
policyTypes:
- Ingress
- Egress
```
Note the fact that there must be no `-` before the podSelector that we added. If we put a `-` then the rule would operate as follows
> **ALLOW** any pod in namespace `default` **OR** any pod in any namespace with label `app=cyan-white-cka28-trb`
That would also permit `cyan-black-cka28-trb` to access, which is incorrect! Without the `-`, the rule operates correctly as follows
> **ALLOW** any pod in namespace `default` **THAT HAS** label `app=cyan-white-cka28-trb`
Which basically means pods in namespace `default` **AND** with correct labels.
Let's test this. We will use the `--connect-timeout` argument for `curl` so as not to wait too long for the expected failed connection from the black pod.
```
k exec -n default cyan-white-cka28-trb -it -- curl --connect-timeout 10 cyan-svc-cka28-trb.cyan-ns-cka28-trb.svc
```
> Output
```html
Welcome to nginx!
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
```
```
k exec -n default cyan-black-cka28-trb -it -- curl --connect-timeout 10 cyan-svc-cka28-trb.cyan-ns-cka28-trb.svc
```
> Output
```
curl: (28) Connection timeout after 10000 ms
command terminated with exit code 28
```
White pod connects, black pod does not - RESULT!
================================================
FILE: docs/16-Ultimate-Mocks/04-Storage/README.md
================================================
# SECTION: Storage
* [10-olive-pvc-cka10-str](./docs/10-CI-olive-pvc-cka10-str.md)
================================================
FILE: docs/16-Ultimate-Mocks/04-Storage/docs/10-CI-olive-pvc-cka10-str.md
================================================
# Cluster 1, Storage, olive-pvc-cka10-str
For this question, please set the context to `cluster1` by running:
```
kubectl config use-context cluster1
```
We want to deploy a python based application on the cluster using a template located at `/root/olive-app-cka10-str`.yaml on `student-node`. However, before you proceed we need to make some modifications to the YAML file as per details given below:
* The YAML should also contain a persistent volume claim with name `olive-pvc-cka10-str` to claim a `100Mi` of storage from `olive-pv-cka10-str PV`.
* Update the deployment to add a sidecar container, which can use `busybox` image (you might need to add a sleep command for this container to keep it running.)
* Share the `python-data` volume with this container and mount the same at path `/usr/src`. Make sure this container only has `read` permissions on this volume.
* Finally, create a pod using this YAML and make sure the POD is in `Running` state.
### Missing from the question, but required to pass
* Create a nodeport service for this deployment with the following specification
* Node port: `32006`
* Name: `olive-svc-cka10-str`
---
### Solution
1. Examine what we have...
```
cat /root/olive-app-cka10-str
```
The PVC volume claim is already present. Look for the PVC
```
kubectl get pvc
```
It is not present, therefore it will have to be created first.
```
kubectl get pv
```
The PV exists. Note the `ACCESS MODES` and `STORAGECLASS`, which are required in the PVC manifest, along with the storage request given in the question.
1. Prepare manfiest for the new PVC
```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: olive-pvc-cka10-str
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 100Mi
storageClassName: olive-stc-cka10-str
```
Then create it. It will not bind yet until the pod is created.
1. Adjust the pod as directed, and add the service to the end.
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: olive-app-cka10-str
spec:
replicas: 1
template:
metadata:
labels:
app: olive-app-cka10-str
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- cluster1-node01
containers:
- name: busybox
image: busybox
command: # <- Any variation of sleep command should work.
- bin/sh # Needs to sleep long enough to get to end of test.
- -c
- sleep 10000
volumeMounts:
- mountPath: /usr/src
name: python-data
readOnly: true
- name: python
image: poroko/flask-demo-app
ports:
- containerPort: 5000
volumeMounts:
- name: python-data
mountPath: /usr/share/
volumes:
- name: python-data
persistentVolumeClaim:
claimName: olive-pvc-cka10-str
selector:
matchLabels:
app: olive-app-cka10-str
---
apiVersion: v1
kind: Service
metadata:
name: olive-svc-cka10-str
namespace: default
spec:
ports:
- nodePort: 32006
port: 5000
protocol: TCP
targetPort: 5000
selector:
app: olive-app-cka10-str
type: NodePort
```
1. Create the resources
```
kubectl apply -f /root/olive-app-cka10-str
```
================================================
FILE: docs/16-Ultimate-Mocks/05-Services-Networking/README.md
================================================
# SECTION: Services/Networking
* [03-C3-External-Webserver](./docs/03-C3-External-Webserver.md)
================================================
FILE: docs/16-Ultimate-Mocks/05-Services-Networking/docs/03-C3-External-Webserver.md
================================================
# Cluster 3 - External Webserver
NOTE: This question is also present in the Ultimate CKAD Mocks. The service name is `external-webserver-ckad01-svcn`, however the solution is exactly the same. If you are doing the CKAD version of this question, put instead `external-webserver-ckad01-svcn` everywhere you see `external-webserver-cka03-svcn`.
For this question, please set the context to cluster3 by running:
```
kubectl config use-context cluster3
```
We have an **external** webserver running on `student-node` which is exposed at port `9999`. We have created a service called `external-webserver-cka03-svcn` that can connect to our local webserver from within the kubernetes cluster3 but at the moment it is not working as expected.
Fix the issue so that other pods within cluster3 can use `external-webserver-cka03-svcn` service to access the webserver.
---
For this we are told that we need to wire up the service to a web server that's running on `student-node` at port `9999`. Let's verify this. On student node run the following
```
curl localhost:9999
```
> Output
```html
Welcome to nginx!
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
```
Yup, it's there!
The important thing to note is that this web server is *outside* of the cluster, therefore the service is going to need to talk to an IP which is not inside the cluster, it is in fact the primary IP address of `student-node`. Let's find that by running the following on `student-node`:
```
ifconfig
```
> Output (note that the values you get for each interface will almost certainly be different)
```
eth0: flags=4163 mtu 1450
inet 192.37.66.3 netmask 255.255.255.0 broadcast 192.37.66.255
ether 02:42:c0:25:42:03 txqueuelen 0 (Ethernet)
RX packets 2179 bytes 713082 (713.0 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2361 bytes 391620 (391.6 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth1: flags=4163 mtu 1500
inet 172.25.0.103 netmask 255.255.255.0 broadcast 172.25.0.255
ether 02:42:ac:19:00:67 txqueuelen 0 (Ethernet)
RX packets 36 bytes 8457 (8.4 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 14 bytes 1593 (1.5 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73 mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1000 (Local Loopback)
RX packets 161 bytes 14395 (14.3 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 161 bytes 14395 (14.3 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
```
The primary interface is `eth0`. Note down the `inet` value for this interface which in this example is `192.37.66.3`. This is the IP address that our service needs to talk to.
If we now do a `kubectl get service` on `external-webserver-cka03-svcn`, we see there's no pod selector and therefore no endpoints. So isn't "working as expected" since it doesn't have any endpoints.
To wire up the service to the external IP, we must explicitly create an endpoint for the service. Note that the name of the endpoint (`metadata.name`) *must exactly match* the name of the service that you want to associate it to. Here's the endpoint with comments indicating what's what.
```yaml
apiVersion: v1
kind: Endpoints
metadata:
name: external-webserver-cka03-svcn # <- Must be same name as the service to associate with
namespace: default
subsets:
- addresses:
- ip: 192.37.66.3 # <- We got this from ifconfig
ports:
- port: 9999 # <- Given in the question
```
Create this in a file and `kubectl apply` it.
Now let's test it using a `wbitt/network-multitool` pod that will contain curl so that we can call the service.
TIP - remember this image! It contains many common networking and DNS tools that can be useful in troubleshooting - and yes it can be used in the real exam.
```
k run test-pod --image wbitt/network-multitool --restart Never -it -- curl external-webserver-cka03-svcn.default.svc
```
> Output
```
The directory /usr/share/nginx/html is not mounted.
Therefore, over-writing the default index.html file with some useful information:
WBITT Network MultiTool (with NGINX) - test-pod - 10.42.0.13 - HTTP: 80 , HTTPS: 443 . (Formerly praqma/network-multitool)
Welcome to nginx!
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.
For online documentation and support please refer to
nginx.org.
Commercial support is available at
nginx.com.
Thank you for using nginx.
```
We got a response - RESULT!
So what we have achieved here is to configure a ClusterIP service that allows pods *inside* the cluster to talk to a service that is *outside* the cluster by way of an explicit endpoint that points to an external IP address. `kube-proxy` takes care of the routing for us.
================================================
FILE: docs/16-Ultimate-Mocks/09-general/README.md
================================================
# General tips
[01-cluster state](./docs/01-cluster-state-questions.md)
================================================
FILE: docs/16-Ultimate-Mocks/09-general/docs/01-cluster-state-questions.md
================================================
# Cluster State Questions
This comes up in both CKA and CKAD tests and is about questions that ask you to write for instance the pods consuming most CPU or most memory to a file, or some other kinds of question such as listing Pod IPs normally using jsonpath or custom columns. These questions are often marked incorrect at the end of the exam and cause much consternation amongst students.
Now the more attentive students may realize why this is the case. If you get such a question near the start of the mock, and then you have questions further on that require you to make deployments into the same cluster, then this is going to change things! The pod that was consuming the most CPU when you answered that question may no longer be the top consumer by the end of the exam, as some newly deployed pod may have a higher CPU usage.
This type of question is about cluster state, and the state of the cluster changes whenever you deploy or delete resources. The marking script can only consider the cluster state after you press `End Exam`. The trick here is to defer answering such questions until you are about to end the exam. Ideally (though this is not always feasible), create a script to answer the question and test it while you are still on the question. Then, when you are about to press the `End Exam` button, run the script again and it will update the file with what the current state is. This should now get the question to pass. In the real exam, you should not have to do this as it is likely that one of the exam clusters is dedicated to such questions so its state won't change by the end of the exam.
## Examples
### Example 1
Store the `pod names` and their `ip addresses` from all namespaces at `/root/pod_ips_ckad02_svcn` where the output is sorted by their IPs.
Please ensure the format as shown below:
```
POD_NAME IP_ADDR
pod-1 ip-1
pod-3 ip-2
pod-2 ip-3
...
```
---
From the required output, this clearly requires Custom Columns
1. Work out the custom columns command to get the required output
Note the use of the `--context` argument here. This ensures the command is run on the correct cluster, irrespective of whether you ran `kubectl config use-context`
```
kubectl --context=cluster3 get pods -A -o custom-columns="POD_NAME:.metadata.name,IP_ADDR:.status.podIP" --sort-by=".status.podIP"
```
1. Adjust this to write to the output file and check the output
```
kubectl --context=cluster3 get pods -A -o custom-columns="POD_NAME:.metadata.name,IP_ADDR:.status.podIP" --sort-by=".status.podIP" > /root/pod_ips_ckad02_svcn
```
Check it
```bash
cat /root/pod_ips_ckad02_svcn
```
1. Now use `vi` to create a file `run-at-end.sh`
```
vi run-at-end.sh
```
Paste the entire kubectl command from above (step 2) into this file. If you have already created this script for a previous similar question, then simply add this line to the file, so the script will answer all such questions when you run it.
1. Test it
```
rm -f /root/pod_ips_ckad02_svcn
source run-at-end.sh
cat /root/pod_ips_ckad02_svcn
```
The output should be the same
1. Finally when you are finished and before pressing `End Exam`, re-run your script
```
source run-at-end.sh
```
### Example 2
Find the pod that consumes the most CPU and store the result to the file `/opt/high_cpu_pod` in the following format
`cluster_name,namespace,pod_name`.
The pod could be in any namespace in any of the clusters that are currently configured on the student-node.
---
Since it says "in any of the clusters", this will really test your skills of bash scripting, plus `kubectl top` has no JSON output option making it even more difficult to script. Having said that, the best way to solve this question is to write the requirements down on your notepad then answer the question manually at the end before you press `End Exam`. Note that you do not need to navigate back to the question to provide the answer - just do it from your notes.
Use a similar approach whether the stat is CPU or memory, or the resource is Pods or Nodes.
* Manual version
1. Get all the cluster names
```
kubectl config get-contexts -o name
```
1. Examine the pod usage on each cluster. Run this command with each value for `--context`
```
kubectl --context=cluster1 top pods -A --sort-by=cpu
```
1. When you have determined the top pod across all clusters, then you can create the output file in vi and manually add the information in the requested format.
* Scripted version
Note - To do it this way would probably take longer than you want to spend unless you're already a shell scripting guru!
```bash
for ctx in $(kubectl config get-contexts -o name)
do
kubectl --context=$ctx top pod --no-headers -A --sort-by=cpu | head -1 | awk -v ctx=$ctx '{printf "%s,%s,%s,%s\n", ctx, $1, $2, $3}'
done | sort -t ',' -k4 -h | tail -1 | sed -E 's/,[0-9]+[a-z]*$//i' > /opt/high_cpu_pod
```
There is a lot going on here, isn't there?
As a working DevOps engineer, this is the sort of thing you would be expected to be able to come up with in your day-to-day job - indeed the lab engineer who developed the marking script for this lab would have to use something like the above! Hence it is important to know how to *use* Linux as well as Kubernetes to be successful in a Kubernetes job. You don't need to know it to Sys Admin level (e.g RHCSA, LFCS).
The following courses are recommended:
* [Linux Basics](https://kodekloud.com/courses/the-linux-basics-course/)
* [Shell Scripts for Beginners](https://kodekloud.com/courses/shell-scripts-for-beginners/)
* [Advanced Bash Scripting](https://kodekloud.com/courses/advanced-bash-scripting/)
So, what is actually going on?
1. The `for` loop lists the cluster contexts one by one storing the cluster name in the variable `ctx`
1. With each context, the `kubectl top pods` command is executed with `-A` for all namespaces...
1. `--no-headers` removes the column headers from the output.
1. `--sort-by=cpu` ensures the pod we need from this cluster is the first pod listed. In `kubectl top`, sort order is descending.
1. Then we pipe the output to `head -1` to get only the first line of results (the top pod for this cluster).
1. Then we pipe it to `awk` to format the output close to what we need, passing in the cluster name so we can include it in the output. The output will look like this
```
cluster1,default,frontend-stable-cka05-arch,396m
```
1. After `done` there will be one line like above for each of the clusters. It would look like this, and note they are in cluster order, not CPU usage order:
```text
cluster1,default,frontend-stable-cka05-arch,396m
cluster2,kube-system,kube-apiserver-cluster2-controlplane,43m
cluster3,kube-system,metrics-server-7b67f64457-9cqrd,5m
cluster4,kube-system,kube-apiserver-cluster4-controlplane,32m
```
1. Pipe to `sort` so we get the highest CPU pod *across all clusters* to the end of the list. `sort` works in ascending order.
1. `-t ','` sets the field separator to be comma.
1. `-k4` means sort by the fourth field (the one containing the CPU value).
1. `-h` means "human" sort, taking into account any SI unit (i.e. the `m` for milli-cpu`). The output will now look like this:
```text
cluster3,kube-system,metrics-server-7b67f64457-9cqrd,5m
cluster4,kube-system,kube-apiserver-cluster4-controlplane,32m
cluster2,kube-system,kube-apiserver-cluster2-controlplane,43m
cluster1,default,frontend-stable-cka05-arch,396m
```
1. Pipe to `tail -1` to get the last entry in the sorted list which is the one we need, which will yield
```
cluster1,default,frontend-stable-cka05-arch,396m
```
1. Finally pipe to `sed` to remove the CPU value and only output the first 3 fields as required by the question. The `sed` expression matches comma, followed by one or more digits, followed by zero or more letters, followed by end of line using extended regex (`-E`) and replaces it with an empty string, thus deleting the matched text. This yields the required output:
```
cluster1,default,frontend-stable-cka05-arch
```
Then redirect the output to the requested file.
================================================
FILE: docs/16-Ultimate-Mocks/README.md
================================================
# CKA Ultimate Mocks
NOTE: CKA Ultimate Mocks is a separate course from the main CKA course, and as such requires a separate payment or is included in Pro subscription.
In this section, we will go through some of the most troublesome questions - these being the ones that get the most requests for help on our various forums.
* [Troubleshooting](./02-Troubleshooting/)
* [Storage](./04-Storage/)
* [Services/Networking](./05-Services-Networking/)
* [General](./09-general/)
================================================
FILE: docs/17-tips-and-tricks/README.md
================================================
# Tips and Tricks
In this bonus section we will discuss some useful tips that can be used preparation for the exam
- [01-Server for testing network policies](docs/01-server-for-testing-network-policies.md)
- [02-Client-for-testing-network-things](docs/02-client--for-testing-network-things.md
)
================================================
FILE: docs/17-tips-and-tricks/docs/01-server-for-testing-network-policies.md
================================================
# Servers for testing network policies
Sometimes you may have a question that asks you to block ingress to a pod on all but some specific port. If a pod that meets the port requirement is not already present in the given namespace, then the issue here is "How do I create a pod onto which to attach the netpol that listens on the given port so I can test the policy?". You can't just run an nginx pod as that always listens on port 80. You could configure it otherwise, but that would require you to mount a configmap into the nginx pod containing an alternate config for nginx with the new port number. That's far too much hassle under exam conditions!
## Simple server
Fortunately, the default Python distribution contains a simple server that can have its port number configured from the command line, meaning you can run it imperatively. Let's say the network policy requires blocking all but port 9000. We can start a server test pod to listen on 9000 like so. If it's a different port, just put that port number instead of 9000.
```
kubectl run server --image python --command -- python -m http.server 9000
```
Get the pod's IP address. Using the IP for curl test is quicker than typing out the DNS name.
```
controlplane $ k get pod server -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
server 1/1 Running 0 16s 192.168.1.12 node01
```
Now run a pod with `curl` in and test connection to the server
```
curl 192.168.1.12:9000
```
You should get a response.
Now apply your network policy and test again.
## Slightly more advanced server
Perhaps you want to set up several pods and have each serve a specific message on a configurable port so you can tell them apart by their reponses. We can do that with a pod and a config map for each. The pod is the same each time - except for giving it a unique name and mounting the appropriate config map.
The following simulates a pod found in one of the Killer.sh network policy questions.
1. Create a config map which contains a shell script to run the server on a given port with a given message
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: db1-configmap
data:
entrypoint.sh: |
#!/bin/sh
echo "database one" > index.htm #<- Message
port=1111 #<- Server Port
# Run server...
while true
do
{ echo -ne "HTTP/1.0 200 OK\r\nContent-Length: $(wc -c
```
1. Now run a pod with `curl` in and test connection to the server
```
curl 192.168.1.12:1111
```
## See also
See also [client for testing](./02-client--for-testing-network-things.md)
================================================
FILE: docs/17-tips-and-tricks/docs/02-client--for-testing-network-things.md
================================================
# Client for testing network things
Often you will get questions that require you to test network polices, or look something up in the Kubernetes DNS. There is a one size fits all pod that you can deploy that has all the network testing tools you could possibly want, including
* curl
* nslookup
* netstat
* dig
* telnet
* nc
and many more.
You run it like so. Commit the image name to memory - this image is a lifesaver! There is nothing to stop you using it in the exam.
```
kubectl run tester --image wbitt/network-multitool
```
When the pod is running, you can exec into it and run the commands
```
$ kubectl exec tester -it -- bash
/# curl something
/# nslookup something-else
/# exit
```
Or run the commands directly if you need to send the results to a file
```
$ kubectl exec tester -it -- nslookup my-service.default.svc > /opt/some-file.txt
```
================================================
FILE: images/Readme.md
================================================
# Images
================================================
FILE: kubeadm-clusters/README.md
================================================
# Kubeadm Cluster Installations
In this section we present various labs for building kubeadm clusters
* AWS - Using KodeKloud AWS Playground
In these labs terraform is used to provision the virtual machines, leaving you to configure the cluster.
* [3 Node Cluster](./aws/)
* [5 Node Highly Available Cluster](./aws-ha/)
* Apple Silicon
For users of Mac with M-series processors, this is a lab that provisions a 2 or 3 node cluster (depending on how much memory your Mac has) using [Multipass](https://multipass.run/).
* [2/3 Node Cluster](./apple-silicon/)
* Vagrant and VirtualBox
For users of Windows and Mac x86, provision a 3 node cluster on your laptop.
* [3 Node Cluster](./virtualbox/)
================================================
FILE: kubeadm-clusters/apple-silicon/README.md
================================================
# Installing Kubernetes the kubeadm way on Apple Silicon
Updated March 2024
In here is the setup specific to newer Apple Silicon (M-series processor) Macs. VirtualBox currently does not have good support for these Macs, therefore we will create the Ubuntu virtual machines using Multipass.
Multipass should also work on older Macs (Intel Core processor), as should [VirtualBox](../virtualbox/).
Click [here](./docs/01-prerequisites.md) to get started.
================================================
FILE: kubeadm-clusters/apple-silicon/delete-virtual-machines.sh
================================================
#!/usr/bin/env bash
set -eo pipefail
NUM_WORKER_NODES=2
MEM_GB=$(( $(sysctl hw.memsize | cut -d ' ' -f 2) / 1073741824 ))
[ $MEM_GB -lt 16 ] && NUM_WORKER_NODES=1
workers=$(for n in $(seq 1 $NUM_WORKER_NODES) ; do echo -n "node0$n " ; done)
for n in $workers controlplane
do
multipass stop $n
multipass delete $n
done
multipass purge
echo
echo "You should now remove all the following lines from /var/db/dhcpd_leases"
echo
cat /var/db/dhcpd_leases | egrep -A 5 -B 1 '(controlplane|node01|node02)'
echo
cat < /dev/null
then
echo -e "${RED}'jq' not found. Please install it${NC}"
echo "https://github.com/stedolan/jq/wiki/Installation#macos"
exit 1
fi
if ! command -v multipass > /dev/null
then
echo -e "${RED}'multipass' not found. Please install it${NC}"
echo "https://multipass.run/install"
exit 1
fi
NUM_WORKER_NODES=2
MEM_GB=$(( $(sysctl hw.memsize | cut -d ' ' -f 2) / 1073741824 ))
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )/scripts
VM_MEM_GB=3G
if [ $MEM_GB -lt 8 ]
then
echo -e "${RED}System RAM is ${MEM_GB}GB. This is insufficient to deploy a working cluster.${NC}"
exit 1
fi
if [ $MEM_GB -lt 16 ]
then
echo -e "${YELLOW}System RAM is ${MEM_GB}GB. Deploying only one worker node.${NC}"
NUM_WORKER_NODES=1
VM_MEM_GB=2G
sleep 1
fi
workers=$(for n in $(seq 1 $NUM_WORKER_NODES) ; do echo -n "node0$n " ; done)
# Determine interface for bridge
interface=""
bridge_arg="--bridged"
for iface in $(multipass networks --format json | jq -r '.list[] | .name')
do
if netstat -rn -f inet | grep "^default.*${iface}" > /dev/null
then
interface=$iface
break
fi
done
if [ "$(multipass get local.bridged-network)" = "" ]
then
echo -e "${BLUE}Configuring bridge network...${NC}"
if [ -z "${interface}" ]
then
echo -e "${YELLOW}No suitable interface detected to use as bridge"
echo "Falling back to NAT installation"
echo -e "You will not be able to use your browser to connect to NodePort services.${NC}"
BUILD_MODE="NAT"
bridge_arg=""
else
# Set the bridge
echo -e "${GREEN}Configuring bridge to interface '$(multipass networks | grep ${interface})'${NC}"
multipass set local.bridged-network=${interface}
fi
fi
# If the nodes are running, reset them
if multipass list --format json | jq -r '.list[].name' | egrep '(controlplane|node01|node02)' > /dev/null
then
echo -n -e $RED
read -p "VMs are running. Delete and rebuild them (y/n)? " ans
echo -n -e $NC
[ "$ans" != 'y' ] && exit 1
fi
# Boot the nodes
for node in controlplane $workers
do
if multipass list --format json | jq -r '.list[].name' | grep "$node"
then
echo -e "${YELLOW}Deleting $node${NC}"
multipass delete $node
multipass purge
fi
echo -e "${BLUE}Launching ${node}${NC}"
if ! multipass launch $bridge_arg --disk 10G --memory $VM_MEM_GB --cpus 2 --name $node jammy 2>/dev/null
then
# Did it actually launch?
sleep 1
if [ "$(multipass list --format json | jq -r --arg no $node '.list[] | select (.name == $no) | .state')" != "Running" ]
then
echo -e "${RED}$node failed to start!${NC}"
exit 1
fi
fi
echo -e "${GREEN}$node booted!${NC}"
done
# Create hostfile entries
echo -e "${BLUE}Setting hostnames${NC}"
hostentries=/tmp/hostentries
set -x
network=$(netstat -rn -f inet | grep "^default.*${interface}" | awk '{print $2}' | awk 'BEGIN { FS="." } { printf "%s.%s.%s", $1, $2, $3 }')
[ -f $hostentries ] && rm -f $hostentries
for node in controlplane $workers
do
if [ "$BUILD_MODE" = "BRIDGE" ]
then
ip=$(multipass info $node --format json | jq -r --arg nw $network 'first( .info[] )| .ipv4 | .[] | select(startswith($nw))')
else
ip=$(multipass info $node --format json | jq -r 'first( .info[] | .ipv4[0] )')
fi
echo "$ip $node" >> $hostentries
done
for node in controlplane $workers
do
multipass transfer $hostentries $node:/tmp/
multipass transfer $SCRIPT_DIR/01-setup-hosts.sh $node:/tmp/
multipass exec $node -- /tmp/01-setup-hosts.sh $BUILD_MODE $network
done
echo -e "${GREEN}Done!${NC}"
if [ "$ARG" = "-auto" ]
then
# Set up hosts
echo -e "${BLUE}Setting up common components${NC}"
join_command=/tmp/join-command.sh
for node in controlplane $workers
do
echo -e "${BLUE}- ${node}${NC}"
multipass transfer $hostentries $node:/tmp/
multipass transfer $SCRIPT_DIR/*.sh $node:/tmp/
for script in 02-setup-kernel.sh 03-setup-nodes.sh 04-kube-components.sh
do
multipass exec $node -- /tmp/$script
done
done
echo -e "${GREEN}Done!${NC}"
# Configure control plane
echo -e "${BLUE}Setting up control plane${NC}"
multipass exec controlplane /tmp/05-deploy-controlplane.sh
multipass transfer controlplane:/tmp/join-command.sh $join_command
echo -e "${GREEN}Done!${NC}"
# Configure workers
for n in $workers
do
echo -e "${BLUE}Setting up ${n}${NC}"
multipass transfer $join_command $n:/tmp
multipass exec $n -- sudo $join_command
echo -e "${GREEN}Done!${NC}"
done
fi
================================================
FILE: kubeadm-clusters/apple-silicon/docs/01-prerequisites.md
================================================
# Prerequisites
* Apple Silicon System (M1/M2/M3 etc)
* 8GB RAM (16GB preferred).
* All configurations - One control plane node will be provisioned - `controlplane`
* If you have less than 16GB then only one worker node will be provisioned - `node01`
* If you have 16GB or more then two workers will be provisioned - `node01` and `node02`
You'll need to install the following first.
* Multipass - https://multipass.run/install. Follow the instructions to install it and check it is working properly. You should be able to successfully create a test Ubuntu VM following their instructions. Delete the test VM when you're done.
* JQ - https://github.com/stedolan/jq/wiki/Installation#macos
Additionally
* Your account on your Mac must have admin privilege and be able to use `sudo`
* Clone this repo down to your Mac. Open your Mac's terminal application. All commands in this guide are executed from the terminal.
```bash
mkdir ~/kodekloud
cd ~/kodekloud
git clone https://github.com/kodekloudhub/certified-kubernetes-administrator-course.git
cd certified-kubernetes-administrator-course/kubeadm-clusters/apple-silicon
```
## Virtual Machine Network
Due to how the virtualization works, the networking for each VM requires two network adapters; one used by Multipass and one used by everything else. Kubernetes components may by default bind to the Multipass adapter, which is *not* what we want, therefore we have pre-set an environment variable `PRIMARY_IP` on all VMs which is the IP address that Kubernetes components should be using. In the coming labs you will see this environment variable being used to ensure Kubernetes components bind to the correct network interface.
`PRIMARY_IP` is defined as the IP address of the network interface on the node that is connected to the network having the default gateway, and is the interface that a node will use to talk to the other nodes and also to the internet. In bridge mode, this is also an address on your internal broadband network and is allocated by your broadband router.
### Bridge Networking
The default configuration in this lab is to bring the VMs up on bridged interfaces. What this means is that your Kubernetes nodes will appear as additional machines on your local network, their IP addresses being provided dynamically by your broadband router. This facilitates the use of your browser to connect to any NodePort services you deploy.
Should you have issues deploying bridge networking, please raise a [bug report](https://github.com/kodekloudhub/certified-kubernetes-administrator-course/issues) and include all details including the output of `deploy-virtual-machines`.
Then retry the lab in NAT mode. How to do this is covered in the [next section](./02-compute-resources.md).
### NAT Networking
In NAT configuration, the network on which the VMs run is isolated from your broadband router's network by a NAT gateway managed by the hypervisor. This means that VMs can see out (and connect to Internet), but you can't see in (i.e. use browser to connect to NodePorts). It is currently not possible to set up port forwarding rules in Multipass to facilitate this.
The network used by the VMs is chosen by Multipass.
It is *recommended* that you leave the pod and service networks as the defaults. If you change them then you will also need to edit the Weave networking manifests to accommodate your change.
If you do decide to change any of these, please treat as personal preference and do not raise a pull request.
## Running Commands in Parallel with iterm2
[iterm2](https://iterm2.com/) which is a popular replacement for the standard Mac terminal application can be used to run the same commands on multiple compute instances at the same time. Some labs in this tutorial require running the same commands on multiple compute instances for instance installing the Kubernetes software. In those cases you may consider using iterm2 and splitting a window into multiple panes with *Broadcast input to all panes* enabled to speed up the provisioning process.
*The use of iterm2 is optional and not required to complete this tutorial*.
To set up as per the image above, do the following in iterm2
1. Right click and select split pane horizontally
1. Do this again to create the third pane (if building 2 workers)
1. In each pane, connect to a different node with `Multipass shell`
1. From the `Session` menu at the top, select `Broadcast` -> `Broadcast imput to all panes`. The small icon at the top right of each pane indicates broadcast mode is enabled.
Input typed or pased to one command prompt will be echoed to the others. Remember to turn off broadcast when you have finished a section that applies to all nodes.
Next: [Compute Resources](02-compute-resources.md)
[](#running-commands-in-parallel-with-iterm2)
================================================
FILE: kubeadm-clusters/apple-silicon/docs/02-compute-resources.md
================================================
# Compute Resources
Because we cannot use VirtualBox and are instead using Multipass, [a script is provided](../deploy-virtual-machines.sh) to create the three VMs.
1. Run the VM deploy script from your Mac terminal application
```bash
./deploy-virtual-machines.sh
```
2. Verify you can connect to all three (two if your Mac only has 8GB RAM) VMs:
```bash
multipass shell controlplane
```
You should see a command prompt like `ubuntu@controlplane:~$`
Type the following to return to the Mac terminal
```bash
exit
```
Do this for `node01` and `node02` as well
If you encountered issues starting the VMs, you can try NAT mode. Note that in NAT mode you will not be able to connect to your NodePort services using your browser.
1. Run
```
./delete-virtual-machines.sh
```
1. Edit `deploy-virtual-machines.sh` and change `BUILD_MODE="BRIDGE"` to `BUILD_MODE="NAT"` at line 15.
# Deleting the Virtual Machines
When you have finished with your cluster and want to reclaim the resources, perform the following steps
1. Exit from all your VM sessions
1. Run the [delete script](../delete-virtual-machines.sh) from your Mac terminal application
```bash
./delete-virtual-machines.sh
````
1. Clean stale DHCP leases. Multipass does not do this automatically and if you do not do it yourself you will eventually run out of IP addresses on the multipass VM network.
1. Edit the following
```bash
sudo vi /var/db/dhcpd_leases
```
1. Remove all blocks that look like this, specifically those with `name` set to `controlplane`, `node01`or `node02`
```text
{
name=controlplane
ip_address=192.168.64.4
hw_address=1,52:54:0:78:4d:ff
identifier=1,52:54:0:78:4d:ff
lease=0x65dc3134
}
```
1. Save the file and exit
Next: [Connectivity](./03-connectivity.md)
Prev: [Prerequisites](./01-prerequisites.md)
================================================
FILE: kubeadm-clusters/apple-silicon/docs/03-connectivity.md
================================================
# Connectivity
We will perform most configuration tasks from the `controlplane` node. After the following steps you will be able to use ssh to log into worker nodes from the `controlplane` prompt.
You can if you prefer log into worker nodes from separate terminal sessions using e.g.
```
multipass shell node01
```
## Access all VMs
Here we create an SSH key pair for the `ubuntu` user who we are logged in as. We will copy the public key of this pair to both workers to permit us to use password-less SSH (and SCP) to get from `controlplane` to these other nodes in the context of the `ubuntu` user which exists on all nodes.
Generate Key Pair on `controlplane` node
[//]: # (host:controlplane)
```bash
ssh-keygen
```
Leave all settings to default (just press ENTER at any questions).
Copy the key to the other hosts. For this step please enter `ubuntu` where a password is requested.
The option `-o StrictHostKeyChecking=no` tells it not to ask if you want to connect to a previously unknown host. Not best practice in the real world, but speeds things up here.
```bash
ssh-copy-id -o StrictHostKeyChecking=no ubuntu@node01
```
```bash
ssh-copy-id -o StrictHostKeyChecking=no ubuntu@node02
```
For each host, the output should be similar to this. If it is not, then you may have entered an incorrect password. Retry the step.
```
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'ubuntu@node01'"
and check to make sure that only the key(s) you wanted were added.
```
Next: [Node Setup](../../generic/04-node-setup.md)
Prev: [Compute Resources](02-compute-resources.md)
================================================
FILE: kubeadm-clusters/apple-silicon/scripts/01-setup-hosts.sh
================================================
#!/usr/bin/env bash
BUILD_MODE=$1
NETWORK=$2
# Set hostfile entries
cat /tmp/hostentries | sudo tee -a /etc/hosts
# Export internal IP of primary NIC as an environment variable
if [ "$BUILD_MODE" = "BRIDGE" ]
then
echo "PRIMARY_IP=$(ip route | grep "^default.*${NETWORK}" | awk '{ print $9 }')" | sudo tee -a /etc/environment > /dev/null
else
echo "PRIMARY_IP=$(ip route | grep default | awk '{ print $9 }')" | sudo tee -a /etc/environment > /dev/null
fi
# Enable password auth in sshd so we can use ssh-copy-id
sudo sed -i 's/#PasswordAuthentication/PasswordAuthentication/' /etc/ssh/sshd_config
sudo sed -i 's/KbdInteractiveAuthentication no/KbdInteractiveAuthentication yes/' /etc/ssh/sshd_config
sudo systemctl restart sshd
# Set password for ubuntu user (it's something random by default)
echo 'ubuntu:ubuntu' | sudo chpasswd
================================================
FILE: kubeadm-clusters/apple-silicon/scripts/02-setup-kernel.sh
================================================
#!/usr/bin/env bash
# Step 2 - Set up Operating System Prerequisites
# Load required kernel modules
sudo modprobe overlay
sudo modprobe br_netfilter
# Persist modules between restarts
cat < /dev/null
sudo chmod +x /tmp/join-command.sh
echo "Installing Weave for pod networking"
kubectl apply -f "https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s-1.11.yaml"
for s in $(seq 60 -10 10)
do
echo "Waiting $s seconds for all control plane pods to be running"
sleep 10
done
================================================
FILE: kubeadm-clusters/apple-silicon/scripts/06-deploy-workers.sh
================================================
#!/usr/bin/env bash
================================================
FILE: kubeadm-clusters/apple-silicon/scripts/tmux.conf
================================================
set -g default-shell /usr/bin/bash
set -g mouse on
bind -n C-x setw synchronize-panes
================================================
FILE: kubeadm-clusters/aws/.gitignore
================================================
.terraform/
*.tfstate*
*.lock*
terraform.tfvars
================================================
FILE: kubeadm-clusters/aws/README.md
================================================
# Installing Kubernetes the kubeadm way on AWS EC2
Updated March 2024
This guide shows how to install a 3 node kubeadm cluster on AWS EC2 instances. If using the KodeKloud AWS Playground environment, please ensure you have selected region `us-east-1` (N. Virginia) from the region selection at the top right of the AWS console. To maintain compatibility with the playground permissions, we will use the following EC2 instance configuration.
* Instance type: `t3.medium`
* Operating System: Ubuntu 22.04 (at time of writing)
* Storage: `gp2`, 8GB
Note that this is an exercise in simply getting a cluster running and is a learning exercise only! It will not be suitable for serving workloads to the internet, nor will it be properly secured, otherwise this guide would be three times longer! It should not be used as a basis for building a production cluster.
[Get Started](./docs/01-prerequisites.md)
================================================
FILE: kubeadm-clusters/aws/docs/01-prerequisites.md
================================================
# Prerequisites
* Access to an AWS Account. Either [KodeKloud Playground](https://kodekloud.com/topic/playground-aws/), or your own.
We have pre-set an environment variable `PRIMARY_IP` on all VMs which is the IP address that kube components should be using. `PRIMARY_IP` is defined as the IP address of the network interface on the node that is connected to the network having the default gateway, and is the interface that a node will use to talk to the other nodes. For those interested, this variable is assigned the result of the following command
```bash
ip route | grep default | awk '{ print $9 }'
```
Next: [Compute Resources](02-compute-resources.md)
================================================
FILE: kubeadm-clusters/aws/docs/02-compute-resources.md
================================================
# Provisioning Compute Resources
We will provision the following infrastructure. The infrastructure will be created by Terraform, so as not to spend too much of the lab time just getting that provisioned, and to allow you to focus on the cluster installation.
As can be seen in this diagram, we will create three EC2 instances to form the cluster and a further one `student-node` from which to perform the configuration. We build the infrastructure using Terraform from AWS CloudShell (so you don't have to install Terraform on your workstation), then log into `student-node` which can access the cluster nodes. This relationship between `student-node` and the cluster nodes is similar to CKA Ultimate Mocks and how the real exam works - you start on a separate node (in this case `student-node`), then use SSH to connect to cluster nodes. Note that SSH connections are only possible in the direction of the arrows. It is not possible to SSH from e.g. `controlplane` directly to `node01`. You must `exit` to `student-node` first. This is also how it is in the exam. `student-node` assumes the role of a [bastion host](https://en.wikipedia.org/wiki/Bastion_host).
We will also set up direct connection from your workstation to the node ports of the workers so that you can browse any NodePort services you create (see security below).
Some basic security will be configured:
* Only the `student-node` will be able to access the cluster's API Server, and this is where you will run `kubectl` commands from when the cluster is running.
* Only the `student-node` can SSH to the cluster nodes.
* Ports required by Kubernetes itself (inc. etcd) and Weave CNI will be configured in security groups on the cluster nodes.
Security issues that would make this unsuitable for a genuine production cluster:
* The kube nodes should be on private subnets (no direct access from the Internet) and placed behind a NAT gateway to allow them to download packages, or with a more extreme security posture, completely [airgapped](https://en.wikipedia.org/wiki/Air_gap_(networking)).
* Access to API server and etcd would be more tightly controlled.
* Use of default VPC is not recommended.
* The node ports will be open to the world - i.e. anyone can connect to them.
* A cloud load balancer coupled with an ingress controller would be provisioned to provide ingress to the cluster. It is _definitely_ not recommended to expose the worker nodes' node ports to the Internet as we are doing here!!!
Other things that will be configured by the Terraform code
* Host names set on the nodes: `controlplane`, `node01`, `node02`
* Content of `/etc/hosts` set up on all nodes for easy use of `ssh` command from `student-node`.
* Generation and distribution of a key pair for logging into instances via SSH.
Let's go ahead and get the infrastructure built!
[Click here](https://kodekloud.com/topic/playground-aws/) to start a playground, and click `START LAB` to request a new AWS Cloud Playground instance. After a few seconds, you will receive a URL and your credentials to access AWS Cloud console. Sign into the console.
Note that you must have KodeKloud Pro subscription to run an AWS playground. If you have your own AWS account, this should still work, however you will bear the cost for any resources created until you delete them.
We will run this entire lab in AWS CloudShell which is a Linux terminal you run inside the AWS console and has most of what we need preconfigured, such as git and the AWS credentials needed by Terraform. [Click here](https://us-east-1.console.aws.amazon.com/cloudshell/home?region=us-east-1) to open CloudShell - note that his link will not work until you have signed into the AWS console.
## Install Terraform
From the CloudShell command prompt...
```bash
curl -O https://releases.hashicorp.com/terraform/1.6.2/terraform_1.6.2_linux_amd64.zip
unzip terraform_1.6.2_linux_amd64.zip
mkdir -p ~/bin
mv terraform ~/bin/
terraform version
```
## Clone this repo
```bash
git clone https://github.com/kodekloudhub/certified-kubernetes-administrator-course.git
```
Now change into the `aws/terraform` directory
```bash
cd certified-kubernetes-administrator-course/kubeadm-clusters/aws/terraform
```
## Provision the infrastructure
1. Run the terraform
```bash
terraform init
terraform plan
terraform apply
```
This should take about half a minute. If this all runs correctly, you will see something like the following at the end of all the output. IP addresses _will be different_ for you
```
Apply complete! Resources: 22 added, 0 changed, 0 destroyed.
Outputs:
address_node01 = "44.220.138.27"
address_node02 = "54.167.161.210"
address_student_node = "34.205.252.168"
connect_student_node = < The connection to the server localhost:8080 was refused - did you specify the right host or port?
which is fine, since we haven't installed kubernetes yet.
## Deleting the cluster
If using KodeKloud playground, this isn't strictly necessary as resources will be deleted when the playground ends.
If you are using your own account, this is *crucial* as you will be billed for the resources created until you delete them - unless of course you want to keep it around and pay. Recall that this is *not* a production hardened installation and could pose a security risk to your account if you leave it lying around.
To delete
1. Return to the CloudShell terminal
1. In the same directory where you ran `terraform apply`, run
```
terraform destroy -auto-approve
```
## Notes on the terraform code
Those of you who are also studying our Terraform courses should look at the terraform files and try to understand what is happening here.
One point of note is that for the `node` instances, we create network interfaces for them as separate resources, then attach these ENIs to the instances when they are built. The reason for this is so that the IP addresses of the instances can be known in advance, such that during instance creation `/etc/hosts` may be created by the user_data script.
Next: [Connectivity](./03-connectivity.md)
Prev: [Prerequisites](./01-prerequisites.md)
================================================
FILE: kubeadm-clusters/aws/docs/03-connectivity.md
================================================
# Connectivity
In this lab, you will access the kubernetes nodes by SSH from `student-node` which you should now be logged into.
Check each one. You must `exit` back to `student-node` before trying the next host.
```bash
ssh controlplane
```
```bash
ssh node01
```
```bash
ssh node02
```
Remember to `exit` back to `student-node` each time. ssh access is not configured *between* nodes. This is standard practice when setting up networks, to have one node from which you can get to all the others.
Next: [Node Setup](../../generic/04-node-setup.md)
Prev: [Compute Resources](02-compute-resources.md)
================================================
FILE: kubeadm-clusters/aws/kubeadm-aws.drawio
================================================
================================================
FILE: kubeadm-clusters/aws/terraform/controlplane.sh
================================================
{
# https://www.linkedin.com/pulse/kubernetes-setup-using-kubeadm-aws-ec2-ubuntu-servers-cedric-wanji/
apt-get update
apt-get install -y apt-transport-https ca-certificates curl jq
cat < /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
modprobe overlay
modprobe br_netfilter
# sysctl params required by setup, params persist across reboots
cat < /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
# Apply sysctl params without reboot
sysctl --system
apt-get install -y containerd
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
systemctl restart containerd
KUBE_LATEST=$(curl -L -s https://dl.k8s.io/release/stable.txt | awk 'BEGIN { FS="." } { printf "%s.%s", $1, $2 }')
mkdir -p /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/${KUBE_LATEST}/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/${KUBE_LATEST}/deb/ /" > /etc/apt/sources.list.d/kubernetes.list
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl
crictl config \
--set runtime-endpoint=unix:///run/containerd/containerd.sock \
--set image-endpoint=unix:///run/containerd/containerd.sock
}
{
kubeadm init
kubectl --kubeconfig /etc/kubernetes/admin.conf apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml
}
{
sudo cp /etc/kubernetes/admin.conf .
sudo chmod 666 admin.conf
}
================================================
FILE: kubeadm-clusters/aws/terraform/data.tf
================================================
###############################################################
#
# This file contains configuration for all data source queries
#
###############################################################
# Get the AMI for Ubuntu 22.04
data "aws_ami" "ubuntu" {
most_recent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["099720109477"] # Canonical
}
# Get Public IP of the cloudshell server. This allows us to lock down SSH access
# into the environment from anyone other than yourself, by inserting its public
# IP to a security group ingress rule.
data "localos_public_ip" "cloudshell_ip" {}
# Get the default VPC details
data "aws_vpc" "default_vpc" {
default = true
}
# Get the subnets to use for network interface creation. We will place one node on each subnet.
# Each subnet is in a different AZ which in practice is good for fault tolerance.
data "aws_subnets" "public" {
filter {
name = "vpc-id"
values = [data.aws_vpc.default_vpc.id]
}
filter {
name = "availability-zone"
values = [
"${var.aws_region}a",
"${var.aws_region}b",
"${var.aws_region}c"
]
}
}
================================================
FILE: kubeadm-clusters/aws/terraform/ec2.tf
================================================
###############################################################
#
# This file contains configuration for all EC2 resources
# ENI, EC2 instance, key-pair, ENI->Security group associations
# and key pair for logging in.
#
###############################################################
# Create an SSH key pair for logging into the EC2 instances
# Security note:
# Generally not good practice to generate keys like this in Terraform
# as the key material is stored in the state file.
# Key pairs should be created externally and passed to Terraform as
# a variable.
resource "tls_private_key" "key_pair" {
algorithm = "RSA"
rsa_bits = 4096
}
# Save the private key to local .ssh directory so it can be used by SSH clients
resource "local_sensitive_file" "pem_file" {
filename = pathexpand("~/.ssh/id_rsa")
file_permission = "600"
content = tls_private_key.key_pair.private_key_pem
}
# Upload the public key of the key pair to AWS so it can be added to the instances
resource "aws_key_pair" "kube_kp" {
key_name = "kube_kp"
public_key = trimspace(tls_private_key.key_pair.public_key_openssh)
}
# Create 3 network interfaces (ENIs) which will be for the 3 cluster nodes
# We need to create these spearately from the instances themselves to prevent
# a circular dependency when setting up host files in the EC2 instances - we
# need to know the IP addresses the nodes will have before they are actually
# created.
resource "aws_network_interface" "kubenode" {
for_each = { for idx, inst in local.instances : inst => idx }
subnet_id = data.aws_subnets.public.ids[each.value]
security_groups = [aws_security_group.egress_all.id]
tags = {
Name = local.instances[each.value]
}
}
# Create the kube node instances
# The user_data will set the hostname and entries for
# all nodes in /etc/hosts
resource "aws_instance" "kubenode" {
for_each = toset(local.instances)
ami = data.aws_ami.ubuntu.image_id
key_name = aws_key_pair.kube_kp.key_name
instance_type = "t3.medium"
network_interface {
device_index = 0
network_interface_id = aws_network_interface.kubenode[each.value].id
}
tags = {
"Name" = each.value
}
user_data = <<-EOT
#!/usr/bin/env bash
hostnamectl set-hostname ${each.value}
cat <> /etc/hosts
${aws_network_interface.kubenode["controlplane"].private_ip} controlplane
${aws_network_interface.kubenode["node01"].private_ip} node01
${aws_network_interface.kubenode["node02"].private_ip} node02
EOF
echo "PRIMARY_IP=$(ip route | grep default | awk '{ print $9 }')" >> /etc/environment
EOT
}
# Create the student_node
# The user_data will set the hostname and entries for
# all nodes in /etc/hosts
resource "aws_instance" "student_node" {
ami = data.aws_ami.ubuntu.image_id
instance_type = "t3.small"
key_name = aws_key_pair.kube_kp.key_name
vpc_security_group_ids = [
aws_security_group.student_node.id,
aws_security_group.egress_all.id
]
tags = {
"Name" = "student_node"
}
user_data = <<-EOT
#!/usr/bin/env bash
hostnamectl set-hostname "student-node"
echo "${tls_private_key.key_pair.private_key_pem}" > /home/ubuntu/.ssh/id_rsa
chown ubuntu:ubuntu /home/ubuntu/.ssh/id_rsa
chmod 600 /home/ubuntu/.ssh/id_rsa
curl -sS https://starship.rs/install.sh | sh -s -- -y
echo 'eval "$(starship init bash)"' >> /home/ubuntu/.bashrc
cat <> /etc/hosts
${aws_network_interface.kubenode["controlplane"].private_ip} controlplane
${aws_network_interface.kubenode["node01"].private_ip} node01
${aws_network_interface.kubenode["node02"].private_ip} node02
EOF
EOT
}
# Attach controlplane security group to controlplane ENI
resource "aws_network_interface_sg_attachment" "controlplane_sg_attachment" {
security_group_id = aws_security_group.controlplane.id
network_interface_id = aws_instance.kubenode["controlplane"].primary_network_interface_id
}
resource "aws_network_interface_sg_attachment" "controlplane_sg_attachment_weave" {
security_group_id = aws_security_group.weave.id
network_interface_id = aws_instance.kubenode["controlplane"].primary_network_interface_id
}
# Attach workernodes security group to node01 ENI
resource "aws_network_interface_sg_attachment" "node01_sg_attachment" {
security_group_id = aws_security_group.workernode.id
network_interface_id = aws_instance.kubenode["node01"].primary_network_interface_id
}
resource "aws_network_interface_sg_attachment" "node01_sg_attachment_weave" {
security_group_id = aws_security_group.weave.id
network_interface_id = aws_instance.kubenode["node01"].primary_network_interface_id
}
# Attach workernodes security group to node02 ENI
resource "aws_network_interface_sg_attachment" "node02_sg_attachment" {
security_group_id = aws_security_group.workernode.id
network_interface_id = aws_instance.kubenode["node02"].primary_network_interface_id
}
resource "aws_network_interface_sg_attachment" "node02_sg_attachment_weave" {
security_group_id = aws_security_group.weave.id
network_interface_id = aws_instance.kubenode["node02"].primary_network_interface_id
}
================================================
FILE: kubeadm-clusters/aws/terraform/main.tf
================================================
###############################################################
#
# This file contains the provide decalarations and outputs
#
###############################################################
terraform {
required_providers {
localos = {
source = "fireflycons/localos"
version = "0.1.2"
}
}
}
provider "aws" {
region = var.aws_region
default_tags {
tags = {
"tf:stackid" = "kubeadm-cluster"
}
}
}
output "connect_student_node" {
description = "SSH command for student-node"
value = <<-EOT
Use the following command to log into student-node
ssh ubuntu@${aws_instance.student_node.public_ip}
You should wait till all instances are fully ready in the EC2 console.
The Status Check colunm should contain "2/2 checks passed"
EOT
}
output "address_student_node" {
description = "Public IP of student-node"
value = aws_instance.student_node.public_ip
}
# We can use either of these IPs to connect to node ports
output "address_node01" {
description = "Public IP of node01"
value = aws_instance.kubenode["node01"].public_ip
}
output "address_node02" {
description = "Public IP of node02"
value = aws_instance.kubenode["node02"].public_ip
}
================================================
FILE: kubeadm-clusters/aws/terraform/security_groups.tf
================================================
###############################################################
#
# This file contains configuration for all security groups
# we will need.
#
###############################################################
# Security group for egress to anywhere.
# Will be applied to all EC2 instances
resource "aws_security_group" "egress_all" {
name = "egress_all"
vpc_id = data.aws_vpc.default_vpc.id
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_security_group" "ingress_vpc" {
name = "ingress_vpc"
vpc_id = data.aws_vpc.default_vpc.id
ingress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = [data.aws_vpc.default_vpc.cidr_block]
}
}
# Security group for ingress to student_node host.
# Permits only cloudshell and EC2 instance connect to connect to it
resource "aws_security_group" "student_node" {
name = "student_node"
vpc_id = data.aws_vpc.default_vpc.id
ingress {
description = "Login SSH"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = [
data.localos_public_ip.cloudshell_ip.cidr
]
}
ingress {
description = "EC2 Instance Connect"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = [
"18.206.107.24/29"
]
}
}
# Security group for ingress to controlplane
resource "aws_security_group" "controlplane" {
name = "controlplane"
vpc_id = data.aws_vpc.default_vpc.id
ingress {
# Allow SSH from any host that has student_node security group
description = "Login SSH"
from_port = 22
to_port = 22
protocol = "tcp"
security_groups = [
aws_security_group.student_node.id
]
}
ingress {
# Allow API server access from anywhere inside the VPC
description = "API Server"
from_port = 6443
to_port = 6443
protocol = "tcp"
cidr_blocks = [
data.aws_vpc.default_vpc.cidr_block
]
}
ingress {
# Allow etcd access from anywhere inside the VPC
description = "etcd"
from_port = 2379
to_port = 2380
protocol = "tcp"
cidr_blocks = [
data.aws_vpc.default_vpc.cidr_block
]
}
}
# Security group for ingress to worker nodes
resource "aws_security_group" "workernode" {
name = "workernode"
vpc_id = data.aws_vpc.default_vpc.id
ingress {
# Allow SSH from any host that has student_node security group
description = "Login SSH"
from_port = 22
to_port = 22
protocol = "tcp"
security_groups = [
aws_security_group.student_node.id
]
}
ingress {
# Allow SSH from any host that has controlplane security group
description = "kubelet api"
from_port = 10250
to_port = 10250
protocol = "tcp"
security_groups = [
aws_security_group.controlplane.id
]
}
ingress {
# Allow access to node ports from your workstation
# and student node
description = "Node Ports"
from_port = 30000
to_port = 32767
protocol = "tcp"
cidr_blocks = [
# Note - insecure. Anyone can connect to node ports.
"0.0.0.0/0"
]
}
}
# Security group for communication between weave pods
resource "aws_security_group" "weave" {
name = "weave"
vpc_id = data.aws_vpc.default_vpc.id
ingress {
description = "Weave TCP"
from_port = 6783
to_port = 6783
protocol = "tcp"
security_groups = [
aws_security_group.controlplane.id,
aws_security_group.workernode.id
]
}
ingress {
description = "Weave UDP"
from_port = 6783
to_port = 6784
protocol = "udp"
security_groups = [
aws_security_group.controlplane.id,
aws_security_group.workernode.id
]
}
}
================================================
FILE: kubeadm-clusters/aws/terraform/variables.tf
================================================
# Region to build in
variable "aws_region" {
type = string
default = "us-east-1"
}
# Names of the EC2 instances to create
locals {
instances = [
"controlplane",
"node01",
"node02"
]
}
================================================
FILE: kubeadm-clusters/aws-ha/.gitignore
================================================
.terraform/
*.tfstate*
*.lock*
terraform.tfvars
================================================
FILE: kubeadm-clusters/aws-ha/README.md
================================================
# Kubeadm HA on AWS EC2
Updated March 2024
This guide shows how to install a 5 node highly-available kubeadm cluster on AWS EC2 instances. If using the KodeKloud AWS Playground environment, please ensure you have selected region `us-east-1` (N. Virginia) from the region selection at the top right of the AWS console. To maintain compatibility with the playground permissions, we will use the following EC2 instance configuration.
* Instance type: `t3.medium`
* Operating System: Ubuntu 22.04 (at time of writing)
* Storage: `gp2`, 8GB
Note that this is an exercise in simply getting a cluster running and is a learning exercise only! It will not be suitable for serving workloads to the internet, nor will it be properly secured, otherwise this guide would be three times longer! It should not be used as a basis for building a production cluster.
[Get Started](./docs/01-prerequisites.md)
================================================
FILE: kubeadm-clusters/aws-ha/docs/01-prerequisites.md
================================================
# Prerequisites
* Access to an AWS Account. Either [KodeKloud Playground](https://kodekloud.com/topic/playground-aws/), or your own.
We have pre-set an environment variable `PRIMARY_IP` on all VMs which is the IP address that kube components should be using. `PRIMARY_IP` is defined as the IP address of the network interface on the node that is connected to the network having the default gateway, and is the interface that a node will use to talk to the other nodes. For those interested, this variable is assigned the result of the following command
```bash
ip route | grep default | awk '{ print $9 }'
```
Next: [Compute Resources](02-compute-resources.md)
================================================
FILE: kubeadm-clusters/aws-ha/docs/02-compute-resources.md
================================================
# Provisioning Compute Resources
We will provision the following infrastructure. The infrastructure will be created by Terraform, so as not to spend too much of the lab time just getting that provisioned, and to allow you to focus on the cluster installation.
As can be seen in this diagram, we will create three EC2 instances to form the cluster and a further one `student-node` from which to perform the configuration. We build the infrastructure using Terraform from AWS CloudShell (so you don't have to install Terraform on your workstation), then log into `student-node` which can access the cluster nodes. This relationship between `student-node` and the cluster nodes is similar to CKA Ultimate Mocks and how the real exam works - you start on a separate node (in this case `student-node`), then use SSH to connect to cluster nodes. Note that SSH connections are only possible in the direction of the arrows. It is not possible to SSH from e.g. `controlplane` directly to `node01`. You must `exit` to `student-node` first. This is also how it is in the exam. `student-node` assumes the role of a [bastion host](https://en.wikipedia.org/wiki/Bastion_host).
We will also set up direct connection from your workstation to the node ports of the workers so that you can browse any NodePort services you create (see security below).
Some basic security will be configured:
* Only the `student-node` will be able to access the cluster's API Server, and this is where you will run `kubectl` commands from when the cluster is running.
* Only the `student-node` can SSH to the cluster nodes.
* Ports required by Kubernetes itself (inc. etcd) and Calico CNI will be configured in security groups on the cluster nodes.
Security issues that would make this unsuitable for a genuine production cluster:
* The kube nodes should be on private subnets (no direct access from the Internet) and placed behind a NAT gateway to allow them to download packages, or with a more extreme security posture, completely [airgapped](https://en.wikipedia.org/wiki/Air_gap_(networking)).
* Access to API server and etcd would be more tightly controlled.
* Use of default VPC is not recommended.
* The node ports will be open to the world - i.e. anyone can connect to them.
* A cloud load balancer coupled with an ingress controller would be provisioned to provide ingress to the cluster. It is _definitely_ not recommended to expose the worker nodes' node ports to the Internet as we are doing here!!!
Other things that will be configured by the Terraform code
* Host names set on the nodes: `loadbalancer`, `controlplane01`, `controlplane02`, `controlplane03`, `node01`, `node02`
* Content of `/etc/hosts` set up on all nodes for easy use of `ssh` command from `student-node`.
* Generation and distribution of a key pair for logging into instances via SSH.
Let's go ahead and get the infrastructure built!
[Click here](https://kodekloud.com/topic/playground-aws/) to start a playground, and click `START LAB` to request a new AWS Cloud Playground instance. After a few seconds, you will receive a URL and your credentials to access AWS Cloud console. Sign into the console.
Note that you must have KodeKloud Pro subscription to run an AWS playground. If you have your own AWS account, this should still work, however you will bear the cost for any resources created until you delete them.
We will run this entire lab in AWS CloudShell which is a Linux terminal you run inside the AWS console and has most of what we need preconfigured, such as git and the AWS credentials needed by Terraform. [Click here](https://us-east-1.console.aws.amazon.com/cloudshell/home?region=us-east-1) to open CloudShell - note that his link will not work until you have signed into the AWS console.
## Install Terraform
From the CloudShell command prompt...
```bash
curl -O https://releases.hashicorp.com/terraform/1.6.2/terraform_1.6.2_linux_amd64.zip
unzip terraform_1.6.2_linux_amd64.zip
mkdir -p ~/bin
mv terraform ~/bin/
terraform version
```
## Clone this repo
```bash
git clone https://github.com/kodekloudhub/certified-kubernetes-administrator-course.git
```
Now change into the `aws/terraform` directory
```bash
cd certified-kubernetes-administrator-course/kubeadm-clusters/aws-ha/terraform
```
## Provision the infrastructure
1. Run the terraform
```bash
terraform init
terraform plan
terraform apply
```
This should take about half a minute. If this all runs correctly, you will see something like the following at the end of all the output. IP addresses _will be different_ for you
```
Apply complete! Resources: 43 added, 0 changed, 0 destroyed.
Outputs:
address_node01 = "44.213.90.240"
address_node02 = "54.173.66.210"
address_student_node = "44.199.229.207"
connect_student_node = < The connection to the server localhost:8080 was refused - did you specify the right host or port?
which is fine, since we haven't installed kubernetes yet.
## Deleting the cluster
If using KodeKloud playground, this isn't strictly necessary as resources will be deleted when the playground ends.
If you are using your own account, this is *crucial* as you will be billed for the resources created until you delete them - unless of course you want to keep it around and pay. Recall that this is *not* a production hardened installation and could pose a security risk to your account if you leave it lying around.
To delete
1. Return to the CloudShell terminal
1. In the same directory where you ran `terraform apply`, run
```
terraform destroy -auto-approve
```
## Notes on the terraform code
Those of you who are also studying our Terraform courses should look at the terraform files and try to understand what is happening here.
One point of note is that for the `node` instances, we create network interfaces for them as separate resources, then attach these ENIs to the instances when they are built. The reason for this is so that the IP addresses of the instances can be known in advance, such that during instance creation `/etc/hosts` may be created by the user_data script.
Next: [Connectivity](./03-connectivity.md)
Prev: [Prerequisites](./01-prerequisites.md)
================================================
FILE: kubeadm-clusters/aws-ha/docs/03-connectivity.md
================================================
# Connectivity
In this lab, you will access the kubernetes nodes by SSH from `student-node` which you should now be logged into.
Check each one. You must `exit` back to `student-node` before trying the next host.
```bash
ssh controlplane01
```
```bash
ssh controlplane02
```
```bash
ssh controlplane03
```
```bash
ssh loadbalancer
```
```bash
ssh node01
```
```bash
ssh node02
```
Remember to `exit` back to `student-node` each time. ssh access is not configured *between* nodes. This is standard practice when setting up networks, to have one node from which you can get to all the others.
Next: [Load Balancer Setup](./04-loadbalancer.md)
Prev: [Compute Resources](02-compute-resources.md)
================================================
FILE: kubeadm-clusters/aws-ha/docs/04-loadbalancer.md
================================================
# Configure the load balancer
Now we will install the load balancer that serves as the endpoint for connecting to API server. This will round-robin API server requests between each of the control plane nodes. For this we will use [HAProxy](https://haproxy.org/) in TCP load balancing mode. In this mode it simply forwards all traffic to its back ends (the control planes) without changing it e.g. doing SSL termination.
First, be logged into `student-node` as directed above.
1. Log into the load balancer
```
ssh loadbalancer
```
1. Become root (saves typing `sudo` before every command)
```bash
sudo -i
```
1. Update the apt package index and install packages needed for HAProxy:
```bash
apt-get update
apt-get install -y haproxy
```
1. Using the [dig](https://linux.die.net/man/1/dig) command which is an alternative to `nslookup` and better for scripting with, we can get the *private* IP addresses of the loadbalancer and 3 control planes.
```bash
dig +short loadbalancer
dig +short controlplane01
dig +short controlplane02
dig +short controlplane03
```
**Terminology**
AWS EC2 instances effectively have two IP addresses:
1. Private IP - This is the IP on the AWS subnet where the instance is launched, and is used for EC2 instances to talk to each other.
1. Public IP - Not always assigned, but is the IP used to reach the EC2 instances from the outside world (i.e. your browser). We will see this later in the testing section.
1. Create the HAProxy configuration file
```bash
cat < /etc/haproxy/haproxy.cfg
frontend kubernetes
bind $(dig +short loadbalancer):6443
option tcplog
mode tcp
default_backend kubernetes-control-nodes
backend kubernetes-control-nodes
mode tcp
balance roundrobin
option tcp-check
server controlplane01 $(dig +short controlplane01):6443 check fall 3 rise 2
server controlplane02 $(dig +short controlplane02):6443 check fall 3 rise 2
server controlplane03 $(dig +short controlplane03):6443 check fall 3 rise 2
EOF
```
1. Restart and check haproxy
```bash
systemctl restart haproxy
systemctl status haproxy
```
It should be warning us that no backend is available - which is true because we haven't installed Kubernetes yet!
1. Exit from `sudo` and then back to `student-node`
```bash
exit
exit
```
Next: [Node Setup](./05-node-setup.md)
Prev: [Connectivity](./03-connectivity.md)
================================================
FILE: kubeadm-clusters/aws-ha/docs/05-node-setup.md
================================================
# Node Setup
**NOTE**
In this and the following sections, you will notice some groups of shell commands are enclosed in `{ }`. This is so that you can copy the block from github with the github copy button and paste to your node terminals. Without this, a group of several commands with `sudo` will stop after the first command and you will probably miss this fact, leading to things not working further on.
---
In this section we will configure the nodes and install prerequisites such as the container runtime (`containerd`).
Perform all the following steps on each of `controlplane01`, `controlplane02`, `controlplane03`, `node01` and `node02`.
[//]: # (host:controlplane-node01-node02)
1. Update the apt package index and install packages needed to use the Kubernetes apt repository:
```bash
{
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl
}
```
1. Set up the required kernel modules and make them persistent
```bash
{
cat <
Prev: [Load Balancer Setup](./04-loadbalancer.md)
================================================
FILE: kubeadm-clusters/aws-ha/docs/06-controlplane.md
================================================
# Boot up controlplane
To create a highly available control plane, we install kubeadm on the first control plane node almost the same way as for a single control plane cluster, then we *join* the other control plane nodes in a similar manner to joining worker nodes
## controlplane01
1. ssh to `controlplane01`
```bash
ssh controlplane01
```
1. Become root
```bash
sudo -i
```
1. Set shell variable for the pod network CIDR.
```bash
POD_CIDR=192.168.0.0/16
```
1. Boot the first control plane using the IP address of the load balancer as the control plane endpoint
Set a shell variable to the IP of the loadbalancer
```bash
LOADBALANCER=$(dig +short loadbalancer)
```
...and install Kubernetes using `--control-plane-endpoint` and `--upload-certs` which instructs it that we are building for multiple controlplane nodes.
```bash
kubeadm init --pod-network-cidr $POD_CIDR \
--control-plane-endpoint ${LOADBALANCER}:6443 \
--upload-certs
```
Copy both join commands that are printed to a notepad for use on other control nodes and the worker nodes.
1. Install network plugin (calico). Weave does not work too well with HA clusters.
```bash
kubectl --kubeconfig /etc/kubernetes/admin.conf create -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.3/manifests/tigera-operator.yaml
kubectl --kubeconfig /etc/kubernetes/admin.conf create -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.3/manifests/custom-resources.yaml
```
1. Check we are up and running
```bash
kubectl --kubeconfig /etc/kubernetes/admin.conf get pods -n kube-system
```
1. Exit root shell
```bash
exit
```
1. Prepare the kubeconfig file for copying to `student-node` node, which is where we will run future `kubectl` commands from.
```bash
{
sudo cp /etc/kubernetes/admin.conf .
sudo chmod 666 admin.conf
}
```
1. Exit to `student-node`
```bash
exit
```
1. On `student-node`, Copy down the kubeconfig so we can run kubectl commands from `student-node`
```bash
{
mkdir ~/.kube
scp controlplane01:~/admin.conf ~/.kube/config
}
## controlplane02 and controlplane03
Be on `student-node`
For each of `controlplane02` and `controlplane03`
1. SSH to `controlplane02`
1. Become root
```bash
sudo -i
```
1. Paste the join command for *control* nodes that was output by `kubeadm init` on `controlplane01`
1. Exit back to `student-node`
```bash
exit
exit
```
1. Repeat the steps 2,3 and 4 on `controlplane03`
Next: [Worker setup](./07-workers.md)
Prev: [Node Setup](./05-node-setup.md)
================================================
FILE: kubeadm-clusters/aws-ha/docs/07-workers.md
================================================
# Join the worker nodes
1. SSH to `node01`
1. Become root
```bash
sudo -i
```
1. Paste the join command for *worker* nodes that was output by `kubeadm init` on `controlplane01`
1. Return to `student-node`
```
exit
exit
```
1. Repeat the steps 2, 3 and 4 on `node02`
1. Now you should be back on `student-node`. Check all nodes are up
```bash
kubectl get nodes -o wide
```
There should now be 3 control nodes and 2 workers.
Next: [Test](./08-test.md)
Prev: [Control Plane Setup](./06-controlplane.md)
================================================
FILE: kubeadm-clusters/aws-ha/docs/08-test.md
================================================
## Create a test service
Run the following on `student-node`
1. Ensure all calico pods are running. They can take a while to initialise
```bash
watch kubectl get pods -n calico-system
```
Press `CTRL-C` to exit watch when pods are stable
1. All 5 nodes should now be ready:
```
kubectl get nodes
```
1. Deploy and expose an nginx pod
```bash
kubectl create deployment nginx --image nginx:alpine
kubectl expose deploy nginx --type=NodePort --port 80
PORT_NUMBER=$(kubectl get service -l app=nginx -o jsonpath="{.items[0].spec.ports[0].nodePort}")
echo -e "\n\nService exposed on NodePort $PORT_NUMBER"
```
[//]: # (command:kubectl wait deployment -n default nginx --for condition=Available=True --timeout=90s)
2. Hit the new service
```bash
curl http://node01:$PORT_NUMBER
curl http://node02:$PORT_NUMBER
```
Both should return the nginx welcome message as HTML text.
Congratulations! You now have a working HA kubeadm cluster.
## Viewing service with a browser
1. Refer to the outputs of the terraform run and get the IP address for either `node01` or `node02`. If you did not note these down, you can find the IP addresses from the [EC2 console](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:) by looking in the `Public IPv4 ...` column (scroll right if this column is not in view).
1. Form a URL using the IP address (it will be different for you) and the NodePort number output from step 1 above, e.g.
```
http://54.167.161.210:32182
```
1. Paste URL to your browser
================================================
FILE: kubeadm-clusters/aws-ha/kubeadm-aws-ha.drawio
================================================
================================================
FILE: kubeadm-clusters/aws-ha/terraform/controlplane.sh
================================================
{
# https://www.linkedin.com/pulse/kubernetes-setup-using-kubeadm-aws-ec2-ubuntu-servers-cedric-wanji/
apt-get update
apt-get install -y apt-transport-https ca-certificates curl jq telnet
cat < /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
modprobe overlay
modprobe br_netfilter
# sysctl params required by setup, params persist across reboots
cat < /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
# Apply sysctl params without reboot
sysctl --system
# Network manager config for calico
mkdir -p /etc/NetworkManager/conf.d/
cat < /etc/NetworkManager/conf.d/calico.conf
[keyfile]
unmanaged-devices=interface-name:cali*;interface-name:tunl*;interface-name:vxlan.calico;interface-name:vxlan-v6.calico;interface-name:wireguard.cali;interface-name:wg-v6.cali
EOF
apt-get install -y containerd
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
systemctl restart containerd
KUBE_LATEST=$(curl -s https://api.github.com/repos/kubernetes/kubernetes/releases | jq -r '.[] | .tag_name' | sed '/-/!{s/$/_/}' | sort -r -V | sed 's/_$//' | fgrep -v '-' | head -1 | awk 'BEGIN { FS="." } { printf "%s.%s", $1, $2 }')
mkdir -p /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/${KUBE_LATEST}/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/${KUBE_LATEST}/deb/ /" > /etc/apt/sources.list.d/kubernetes.list
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl
crictl config \
--set runtime-endpoint=unix:///run/containerd/containerd.sock \
--set image-endpoint=unix:///run/containerd/containerd.sock
[[ "$(hostname)" == controlplane* ]] && kubeadm config images pull
if [ "$(hostname)" == "controlplane01" ]
then
curl -L https://github.com/projectcalico/calico/releases/download/v3.26.3/calicoctl-linux-amd64 -o calicoctl
chmod +x ./calicoctl
mv ./calicoctl /usr/local/bin
fi
}
# control1
{
kubeadm init --control-plane-endpoint $(dig +short loadbalancer):6443 --upload-certs --pod-network-cidr=192.168.0.0/16
kubectl --kubeconfig /etc/kubernetes/admin.conf create -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.3/manifests/tigera-operator.yaml
kubectl --kubeconfig /etc/kubernetes/admin.conf create -f https://raw.githubusercontent.com/projectcalico/calico/v3.26.3/manifests/custom-resources.yaml
}
{
sudo cp /etc/kubernetes/admin.conf .
sudo chmod 666 admin.conf
}
# HAProxy
{
apt-get update
apt-get install -y haproxy
cat < /etc/haproxy/haproxy.cfg
frontend kubernetes
bind $(dig +short loadbalancer):6443
option tcplog
mode tcp
default_backend kubernetes-control-nodes
backend kubernetes-control-nodes
mode tcp
balance roundrobin
option tcp-check
server controlplane01 $(dig +short controlplane01):6443 check fall 3 rise 2
server controlplane02 $(dig +short controlplane02):6443 check fall 3 rise 2
server controlplane03 $(dig +short controlplane03):6443 check fall 3 rise 2
EOF
systemctl restart haproxy
}
================================================
FILE: kubeadm-clusters/aws-ha/terraform/data.tf
================================================
###############################################################
#
# This file contains configuration for all data source queries
#
###############################################################
# Get the AMI for Ubuntu 22.04
data "aws_ami" "ubuntu" {
most_recent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["099720109477"] # Canonical
}
# Get Public IP of the cloudshell server. This allows us to lock down SSH access
# into the environment from anyone other than yourself, by inserting its public
# IP to a security group ingress rule.
data "localos_public_ip" "cloudshell_ip" {}
# Get local folders (for storing ssh keys)
data "localos_folders" "folders" {}
# Get the default VPC details
data "aws_vpc" "default_vpc" {
default = true
}
# Get the subnets to use for network interface creation. We will place one node on each subnet.
# Each subnet is in a different AZ which in practice is good for fault tolerance.
data "aws_subnets" "public" {
filter {
name = "vpc-id"
values = [data.aws_vpc.default_vpc.id]
}
filter {
name = "availability-zone"
values = [
"${var.aws_region}a",
"${var.aws_region}b",
"${var.aws_region}c"
]
}
}
================================================
FILE: kubeadm-clusters/aws-ha/terraform/ec2.tf
================================================
###############################################################
#
# This file contains configuration for all EC2 resources
# ENI, EC2 instance, key-pair, ENI->Security group associations
# and key pair for logging in.
#
###############################################################
# Create an SSH key pair for logging into the EC2 instances
# Security note:
# Generally not good practice to generate keys like this in Terraform
# as the key material is stored in the state file.
# Key pairs should be created externally and passed to Terraform as
# a variable.
resource "tls_private_key" "key_pair" {
algorithm = "RSA"
rsa_bits = 4096
}
# Save the private key to local .ssh directory so it can be used by SSH clients
resource "local_sensitive_file" "pem_file" {
filename = "${data.localos_folders.folders.ssh}/id_rsa"
file_permission = "600"
content = tls_private_key.key_pair.private_key_pem
}
# Upload the public key of the key pair to AWS so it can be added to the instances
resource "aws_key_pair" "kube_kp" {
key_name = "kube_kp"
public_key = trimspace(tls_private_key.key_pair.public_key_openssh)
}
# Create one network interfaces (ENI) for each kubenode.
# We need to create these spearately from the instances themselves to prevent
# a circular dependency when setting up host files in the EC2 instances - we
# need to know the IP addresses the nodes will have before they are actually
# created.
resource "aws_network_interface" "kubenode" {
for_each = { for idx, inst in local.instances : inst => idx }
subnet_id = data.aws_subnets.public.ids[each.value % 3]
security_groups = [aws_security_group.egress_all.id]
tags = {
Name = local.instances[each.value]
}
}
# Create the kube node instances
# The user_data will set the hostname and entries for
# all nodes in /etc/hosts
resource "aws_instance" "kubenode" {
for_each = toset(local.instances)
ami = data.aws_ami.ubuntu.image_id
key_name = aws_key_pair.kube_kp.key_name
instance_type = "t3.medium"
network_interface {
device_index = 0
network_interface_id = aws_network_interface.kubenode[each.value].id
}
tags = {
"Name" = each.value
}
user_data = <<-EOT
#!/usr/bin/env bash
hostnamectl set-hostname ${each.value}
cat <> /etc/hosts
${aws_network_interface.kubenode["loadbalancer"].private_ip} loadbalancer
${aws_network_interface.kubenode["controlplane01"].private_ip} controlplane01
${aws_network_interface.kubenode["controlplane02"].private_ip} controlplane02
${aws_network_interface.kubenode["controlplane03"].private_ip} controlplane03
${aws_network_interface.kubenode["node01"].private_ip} node01
${aws_network_interface.kubenode["node02"].private_ip} node02
EOF
echo "PRIMARY_IP=$(ip route | grep default | awk '{ print $9 }')" >> /etc/environment
EOT
}
# Create the student_node
# The user_data will set the hostname and entries for
# all nodes in /etc/hosts
resource "aws_instance" "student_node" {
ami = data.aws_ami.ubuntu.image_id
instance_type = "t3.small"
key_name = aws_key_pair.kube_kp.key_name
vpc_security_group_ids = [
aws_security_group.student_node.id,
aws_security_group.egress_all.id
]
tags = {
"Name" = "student_node"
}
user_data = <<-EOT
#!/usr/bin/env bash
hostnamectl set-hostname "student-node"
echo "${tls_private_key.key_pair.private_key_pem}" > /home/ubuntu/.ssh/id_rsa
chown ubuntu:ubuntu /home/ubuntu/.ssh/id_rsa
chmod 600 /home/ubuntu/.ssh/id_rsa
curl -sS https://starship.rs/install.sh | sh -s -- -y
echo 'eval "$(starship init bash)"' >> /home/ubuntu/.bashrc
cat <> /etc/hosts
${aws_network_interface.kubenode["loadbalancer"].private_ip} loadbalancer
${aws_network_interface.kubenode["controlplane01"].private_ip} controlplane01
${aws_network_interface.kubenode["controlplane02"].private_ip} controlplane02
${aws_network_interface.kubenode["controlplane03"].private_ip} controlplane03
${aws_network_interface.kubenode["node01"].private_ip} node01
${aws_network_interface.kubenode["node02"].private_ip} node02
EOF
EOT
}
# Attach loadbalancer security group to loadbalancer ENI
resource "aws_network_interface_sg_attachment" "loadbalancer_sg_attachment" {
security_group_id = aws_security_group.loadbalancer.id
network_interface_id = aws_instance.kubenode["loadbalancer"].primary_network_interface_id
}
# Attach controlplane and calico security groups to controlplane01 ENI
resource "aws_network_interface_sg_attachment" "controlplane01_sg_attachment" {
security_group_id = aws_security_group.controlplane.id
network_interface_id = aws_instance.kubenode["controlplane01"].primary_network_interface_id
}
resource "aws_network_interface_sg_attachment" "controlplane01_sg_attachment_calico" {
security_group_id = aws_security_group.calico.id
network_interface_id = aws_instance.kubenode["controlplane01"].primary_network_interface_id
}
# Attach controlplane and calico security groups to controlplane02 ENI
resource "aws_network_interface_sg_attachment" "controlplane02_sg_attachment" {
security_group_id = aws_security_group.controlplane.id
network_interface_id = aws_instance.kubenode["controlplane02"].primary_network_interface_id
}
resource "aws_network_interface_sg_attachment" "controlplane02_sg_attachment_calico" {
security_group_id = aws_security_group.calico.id
network_interface_id = aws_instance.kubenode["controlplane02"].primary_network_interface_id
}
# Attach controlplane and calico security groups to controlplane03 ENI
resource "aws_network_interface_sg_attachment" "controlplane03_sg_attachment" {
security_group_id = aws_security_group.controlplane.id
network_interface_id = aws_instance.kubenode["controlplane03"].primary_network_interface_id
}
resource "aws_network_interface_sg_attachment" "controlplane03_sg_attachment_calico" {
security_group_id = aws_security_group.calico.id
network_interface_id = aws_instance.kubenode["controlplane03"].primary_network_interface_id
}
# Attach workernodes and calico security groups to node01 ENI
resource "aws_network_interface_sg_attachment" "node01_sg_attachment" {
security_group_id = aws_security_group.workernode.id
network_interface_id = aws_instance.kubenode["node01"].primary_network_interface_id
}
resource "aws_network_interface_sg_attachment" "node01_sg_attachment_calico" {
security_group_id = aws_security_group.calico.id
network_interface_id = aws_instance.kubenode["node01"].primary_network_interface_id
}
# Attach workernodes and calico security groups to node02 ENI
resource "aws_network_interface_sg_attachment" "node02_sg_attachment" {
security_group_id = aws_security_group.workernode.id
network_interface_id = aws_instance.kubenode["node02"].primary_network_interface_id
}
resource "aws_network_interface_sg_attachment" "node02_sg_attachment_calico" {
security_group_id = aws_security_group.calico.id
network_interface_id = aws_instance.kubenode["node02"].primary_network_interface_id
}
================================================
FILE: kubeadm-clusters/aws-ha/terraform/lb.sh
================================================
{
apt-get update
apt-get install -y haproxy
cat < /etc/haproxy/haproxy.cfg
frontend kubernetes
bind $(dig +short loadbalancer):6443
option tcplog
mode tcp
default_backend kubernetes-control-nodes
backend kubernetes-control-nodes
mode tcp
balance roundrobin
option tcp-check
server control-1 $(dig +short controlplane01):6443 check fall 3 rise 2
server control-2 $(dig +short controlplane02):6443 check fall 3 rise 2
server control-3 $(dig +short controlplane03):6443 check fall 3 rise 2
EOF
systemctl restart haproxy
}
================================================
FILE: kubeadm-clusters/aws-ha/terraform/main.tf
================================================
###############################################################
#
# This file contains the provide decalarations and outputs
#
###############################################################
terraform {
required_providers {
localos = {
source = "fireflycons/localos"
version = "0.1.2"
}
}
}
provider "aws" {
region = var.aws_region
default_tags {
tags = {
"tf:stackid" = "kubeadm-cluster"
}
}
}
output "connect_student_node" {
description = "SSH command for student-node"
value = <<-EOT
Use the following command to log into student-node
ssh ubuntu@${aws_instance.student_node.public_ip}
You should wait till all instances are fully ready in the EC2 console.
The Status Check colunm should contain "2/2 checks passed"
EOT
}
output "address_student_node" {
description = "Public IP of student-node"
value = aws_instance.student_node.public_ip
}
# We can use either of these IPs to connect to node ports
output "address_node01" {
description = "Public IP of node01"
value = aws_instance.kubenode["node01"].public_ip
}
output "address_node02" {
description = "Public IP of node02"
value = aws_instance.kubenode["node02"].public_ip
}
================================================
FILE: kubeadm-clusters/aws-ha/terraform/security_groups.tf
================================================
###############################################################
#
# This file contains configuration for all security groups
# we will need.
#
###############################################################
# Security group for egress to anywhere.
# Will be applied to all EC2 instances
resource "aws_security_group" "egress_all" {
name = "egress_all"
vpc_id = data.aws_vpc.default_vpc.id
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
resource "aws_security_group" "ingress_vpc" {
name = "ingress_vpc"
vpc_id = data.aws_vpc.default_vpc.id
ingress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = [data.aws_vpc.default_vpc.cidr_block]
}
}
# Security group for ingress to student_node host.
# Permits only cloudshell and EC2 instance connect to connect to it
resource "aws_security_group" "student_node" {
name = "student_node"
vpc_id = data.aws_vpc.default_vpc.id
ingress {
description = "Login SSH"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = [
data.localos_public_ip.cloudshell_ip.cidr
]
}
ingress {
description = "EC2 Instance Connect"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = [
"18.206.107.24/29"
]
}
}
# Security group for ingress to loadbalancer host.
# Permits only student-node and EC2 instance connect to SSH to it
# Beacuse the loadbalancer is the ingress to API server, allow
# API server port from VPC
resource "aws_security_group" "loadbalancer" {
name = "loadbalancer"
vpc_id = data.aws_vpc.default_vpc.id
ingress {
# Allow SSH from student-node
description = "Login SSH"
from_port = 22
to_port = 22
protocol = "tcp"
security_groups = [
aws_security_group.student_node.id
]
}
ingress {
description = "EC2 Instance Connect"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = [
"18.206.107.24/29"
]
}
ingress {
# Allow API server access from anywhere in VPC
description = "API Server"
from_port = 6443
to_port = 6443
protocol = "tcp"
cidr_blocks = [
data.aws_vpc.default_vpc.cidr_block
]
}
}
# Security group for ingress to controlplane
resource "aws_security_group" "controlplane" {
name = "controlplane"
vpc_id = data.aws_vpc.default_vpc.id
ingress {
# Allow SSH from student-node
description = "Login SSH"
from_port = 22
to_port = 22
protocol = "tcp"
security_groups = [
aws_security_group.student_node.id
]
}
ingress {
# Allow API server access only from loadbalancer
description = "API Server from LB"
from_port = 6443
to_port = 6443
protocol = "tcp"
security_groups = [
aws_security_group.loadbalancer.id
]
}
ingress {
# Allow etcd access from student-node
# e.g. to run etcdctl
description = "etcd frorm SN"
from_port = 2379
to_port = 2380
protocol = "tcp"
security_groups = [
aws_security_group.student_node.id
]
}
}
# Additional rules for control plane SG to allow
# control plane components to talk to each other
resource "aws_vpc_security_group_ingress_rule" "controlplane_etcd" {
description = "etcd gossip"
ip_protocol = "tcp"
from_port = 2379
to_port = 2380
security_group_id = aws_security_group.controlplane.id
referenced_security_group_id = aws_security_group.controlplane.id
}
resource "aws_vpc_security_group_ingress_rule" "controlplane_scheduler" {
description = "kubeschduler gossip"
ip_protocol = "tcp"
from_port = 10259
to_port = 10259
security_group_id = aws_security_group.controlplane.id
referenced_security_group_id = aws_security_group.controlplane.id
}
resource "aws_vpc_security_group_ingress_rule" "controlplane_cm" {
description = "controller-manager gossip"
ip_protocol = "tcp"
from_port = 10257
to_port = 10257
security_group_id = aws_security_group.controlplane.id
referenced_security_group_id = aws_security_group.controlplane.id
}
# Security group for ingress to worker nodes
resource "aws_security_group" "workernode" {
name = "workernode"
vpc_id = data.aws_vpc.default_vpc.id
ingress {
# Allow SSH from student-node
description = "Login SSH"
from_port = 22
to_port = 22
protocol = "tcp"
security_groups = [
aws_security_group.student_node.id
]
}
ingress {
# Allow SSH from any host that has controlplane security group
description = "kubelet api"
from_port = 10250
to_port = 10250
protocol = "tcp"
security_groups = [
aws_security_group.controlplane.id
]
}
ingress {
# Allow access to node ports from your workstation
# and student node
description = "Node Ports"
from_port = 30000
to_port = 32767
protocol = "tcp"
cidr_blocks = [
# Note - insecure. Anyone can connect to node ports.
"0.0.0.0/0"
]
}
}
# Security group for communication between calico pods
# Applied to all nodes
resource "aws_security_group" "calico" {
name = "calico"
vpc_id = data.aws_vpc.default_vpc.id
}
resource "aws_vpc_security_group_ingress_rule" "calico_bgp" {
description = "bgp"
from_port = 179
to_port = 179
ip_protocol = "tcp"
security_group_id = aws_security_group.calico.id
referenced_security_group_id = aws_security_group.calico.id
}
resource "aws_vpc_security_group_ingress_rule" "calico_ip_in_ip" {
description = "ip-in-ip"
ip_protocol = "4"
security_group_id = aws_security_group.calico.id
referenced_security_group_id = aws_security_group.calico.id
}
resource "aws_vpc_security_group_ingress_rule" "calico_vxlan" {
description = "vxlan"
from_port = 4789
to_port = 4789
ip_protocol = "udp"
security_group_id = aws_security_group.calico.id
referenced_security_group_id = aws_security_group.calico.id
}
resource "aws_vpc_security_group_ingress_rule" "calico_typha" {
description = "typha"
from_port = 5473
to_port = 5473
ip_protocol = "tcp"
security_group_id = aws_security_group.calico.id
referenced_security_group_id = aws_security_group.calico.id
}
resource "aws_vpc_security_group_ingress_rule" "calico_wireguard" {
description = "wireguard"
from_port = 51820
to_port = 51821
ip_protocol = "udp"
security_group_id = aws_security_group.calico.id
referenced_security_group_id = aws_security_group.calico.id
}
# Calico daemonsets seem to require this port
resource "aws_vpc_security_group_ingress_rule" "calico_apiserver" {
description = "api-server gossip"
ip_protocol = "tcp"
from_port = 6443
to_port = 6443
security_group_id = aws_security_group.calico.id
referenced_security_group_id = aws_security_group.calico.id
}
================================================
FILE: kubeadm-clusters/aws-ha/terraform/variables.tf
================================================
# Region to build in
variable "aws_region" {
type = string
default = "us-east-1"
}
# Names of the EC2 instances to create
locals {
instances = [
"loadbalancer",
"controlplane01",
"controlplane02",
"controlplane03",
"node01",
"node02"
]
}
================================================
FILE: kubeadm-clusters/generic/04-node-setup.md
================================================
# Node Setup
**NOTE**
In this and the following sections, you will notice some groups of shell commands are enclosed in `{ }`. This is so that you can copy the block from github with the github copy button and paste to your node terminals. Without this, a group of several commands with `sudo` will stop after the first command and you will probably miss this fact, leading to things not working further on.
---
In this section we will configure the nodes and install prerequisites such as the container runtime (`containerd`).
Perform all the following steps on each of `controlplane`, `node01` and `node02`. For this section you can run the commands simultaneously on all nodes if using iterm2 broadcast (Mac) or tmux (others).
[//]: # (host:controlplane-node01-node02)
1. Update the apt package index and install packages needed to use the Kubernetes apt repository:
```bash
{
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl
}
```
1. Set up the required kernel modules and make them persistent
```bash
{
cat <
Prev: Connectivity ([VirtualBox](../virtualbox/docs/03-connectivity.md)) ([Apple Silicon](../apple-silicon/docs/03-connectivity.md)) [AWS](../aws/docs/03-connectivity.md)
================================================
FILE: kubeadm-clusters/generic/05-controlplane.md
================================================
# Boot the controlplane
[//]: # (host:controlplane)
On the `controlplane` node
1. Set shell variables for the pod and network network CIDRs. The API server advertise address is using the predefined variable described in the [previous section](./04-node-setup.md). Note that with bridged mode virtual machines (the default for this installation) it is important that neither of the following IP ranges overlap your home network which for nearly all broadband router default configurations is `192.168.0.0/24`
```bash
POD_CIDR=10.244.0.0/16
SERVICE_CIDR=10.96.0.0/16
```
1. Start controlplane
Here we are using arguments to `kubeadm` to ensure that it uses the networks and IP address we want rather than choosing defaults which may be incorrect.
```bash
sudo kubeadm init --pod-network-cidr $POD_CIDR --service-cidr $SERVICE_CIDR --apiserver-advertise-address $PRIMARY_IP
```
[//]: # (command:sleep 10)
1. Copy the `kubeadm join` command printed to your notepad to use on the worker nodes.
1. Set up the kubeconfig so it can be used by the user you are logged in as
1. Create directory for kubeconfig, copy the admin kubeconfig as the default kubeconfig for current user (`vagrant` on VirtualBox, `ubuntu` on Multipass and AWS) and set the correct file permissions.Note that in Linux, the command `id -u` returns your user name on the system and `id -g` returns your group name.
```bash
{
mkdir ~/.kube
sudo cp /etc/kubernetes/admin.conf ~/.kube/config
sudo chown $(id -u):$(id -g) ~/.kube/config
chmod 600 ~/.kube/config
}
```
1. Verify
```bash
kubectl get pods -n kube-system
```
1. Install Calico CNI for cluster networking
Calico is a CNI that supports Network Policies. An alternative CNI that's discussed in the course is Flannel, however Flannel does not support Netowrk Policy. It provides only the basic pod networking.
1. Install the Tigera operator and custom resource definitions.
```bash
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.30.1/manifests/tigera-operator.yaml
```
1. Install Calico by creating the necessary custom resources. If using bridge networking for your virtual machines which is the default, we are going to have to make a modification to these custom resources to agree with the POD_CIDR we set above, or else the pod network will overlap your home network and the cluster won't work.
1. Download the custom resource manifest
```bash
curl -LO https://raw.githubusercontent.com/projectcalico/calico/v3.30.1/manifests/custom-resources.yaml
```
1. Use `sed` to change the ipPool network in the manifest from the default of `192.168.0.0/16` to the value of `POD_IP` set above. You could do it by editing with `vi` but `sed` is faster.
```bash
sed -i "s#192.168.0.0/16#$POD_CIDR#" custom-resources.yaml
```
1. Apply the edited manifest
```bash
kubectl apply -f custom-resources.yaml
```
It will take a minute or two for all the calico resources to be up and running. You can monitor the progress with the following command
```
watch kubectl get tigerastatus
```
Wait until all items in the list have a `True` status for available, then press `CTRL-C` to exit the `watch` command.
[Calico Installation Documentation](https://docs.tigera.io/calico/latest/getting-started/kubernetes/quickstart)
[//]: # (command:sleep 120)
1. Verify controlplane
```bash
kubectl get pods -n kube-system
```
[Kubernetes Documentation Link](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/)
Next: [Join Workers](./06-workers.md)
Prev: [Node setup](./04-node-setup.md)
================================================
FILE: kubeadm-clusters/generic/06-workers.md
================================================
# Boot the worker nodes
Here we will join the worker nodes to the cluster. You will need the `kubeadm join` command from the previous step
Ref: https://stackoverflow.com/questions/59629319/unable-to-upgrade-connection-pod-does-not-exist
## Join Workers
If you did not note down the join command on the controlplane node after running `kubeadm`, you can recover it by running the following on `controlplane`
```bash
kubeadm token create --print-join-command
```
[//]: # (host:node01-node02)
[//]: # (comment:Run kubeadm join)
On each of `node01` and `node02` do the following
1. Become root (if you are not already)
```
sudo -i
```
1. Join the node
> Paste the `kubeadm join` command output by `kubeadm init` on the control plane
### Verify
On `controlplane` run the following. After a few seconds both nodes should be ready
```
kubectl get nodes
```
[Kubernetes Documentation Link](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/adding-linux-nodes/)
Next: [Test](./07-test.md)
Prev: [Boot controlplane](./05-controlplane.md)
================================================
FILE: kubeadm-clusters/generic/07-test.md
================================================
# Test the cluster
Here we will test the cluster by creating a workload with a node port service and get the results using `curl`
Do the following on `controlplane`
[//]: # (host:controlplane)
1. Deploy and expose an nginx pod
```bash
kubectl create deployment nginx --image nginx:alpine
kubectl expose deploy nginx --type=NodePort --port 80
PORT_NUMBER=$(kubectl get service -l app=nginx -o jsonpath="{.items[0].spec.ports[0].nodePort}")
echo -e "\n\nService exposed on NodePort $PORT_NUMBER"
```
[//]: # (command:kubectl wait deployment -n default nginx --for condition=Available=True --timeout=90s)
2. Hit the new service
```bash
curl http://node01:$PORT_NUMBER
curl http://node02:$PORT_NUMBER
```
If you get this error:
```
curl: (7) Failed to connect to node01 port 30659 after 0 ms: Connection refused
```
Wait a few seconds and try again. Both should return the nginx welcome message as HTML text.
Congratulations! You now have a working kubeadm cluster.
## Viewing service with a browser
### VirtualBox or Apple Silicon
If you installed the cluster with bridge networking (the default), then you can view NodePort services with your browser.
Run the following command on `controlplane` to get browser address, then copy the output to your browser:
```bash
echo "http://$(dig +short node01):$PORT_NUMBER"
```
### AWS
1. Refer to the outputs of the terraform run and get the IP address for either `node01` or `node02`. If you did not note these down, you can find the IP addresses from the [EC2 console](https://us-east-1.console.aws.amazon.com/ec2/home?region=us-east-1#Instances:) by looking in the `Public IPv4 ...` column (scroll right if this column is not in view).
1. Form a URL using the IP address (it will be different for you) and the NodePort number output from step 1 above, e.g.
```
http://54.167.161.210:32182
```
1. Paste URL to your browser
You may also copy the KUBECONFIG to `student-node` thus allowing you to run `kubectl` commands from there:
1. Return to `student-node`.
1. Run the following:
```bash
mkdir ~/.kube
scp controlplane:~/.kube/config ~/.kube/config
kubectl get pods -n kube-system
```
Prev: [Worker nodes](./06-workers.md)
================================================
FILE: kubeadm-clusters/generic/README.md
================================================
# kubeadm-generic
This folder contains the generic steps to setting up a Kubeadm cluster irrespective of the virtual machine environment you are using.
All the 3-node cluster builds link to here.
## Start Points
* [VirtualBox](../virtualbox/)
* [Apple Silicon](../apple-silicon/)
* [AWS](../aws/)
================================================
FILE: kubeadm-clusters/virtualbox/.gitignore
================================================
quick-steps/
================================================
FILE: kubeadm-clusters/virtualbox/README.md
================================================
# Installing Kubernetes the kubeadm way on VirtualBox
Updated March 2024
[Click here](./docs/01-prerequisites.md) to begin.
================================================
FILE: kubeadm-clusters/virtualbox/Vagrantfile
================================================
# -*- mode: ruby -*-
# vi:set ft=ruby sw=2 ts=2 sts=2:
# Set the build mode
# "BRIDGE" - Places VMs on your local network so cluster can be accessed from browser.
# You must have enough spare IPs on your network for the cluster nodes.
# "NAT" - Places VMs in a private virtual network. Cluster cannot be accessed
# without setting up a port forwarding rule for every NodePort exposed.
# Use this mode if for some reason BRIDGE doesn't work for you.
BUILD_MODE = "BRIDGE"
# Define the number of worker nodes
# If this number is changed, remember to update setup-hosts.sh script with the new hosts IP details in /etc/hosts of each VM.
NUM_WORKER_NODES = 2
# Network parameters for NAT mode
IP_NW = "192.168.56"
MASTER_IP_START = 11
NODE_IP_START = 20
# Host operating sysem detection
module OS
def OS.windows?
(/cygwin|mswin|mingw|bccwin|wince|emx/ =~ RUBY_PLATFORM) != nil
end
def OS.mac?
(/darwin/ =~ RUBY_PLATFORM) != nil
end
def OS.unix?
!OS.windows?
end
def OS.linux?
OS.unix? and not OS.mac?
end
def OS.jruby?
RUBY_ENGINE == "jruby"
end
end
# Determine host adpater for bridging in BRIDGE mode
def get_bridge_adapter()
if OS.windows?
return %x{powershell -Command "Get-NetRoute -DestinationPrefix 0.0.0.0/0 | Get-NetAdapter | Select-Object -ExpandProperty InterfaceDescription"}.chomp
elsif OS.linux?
return %x{ip route | grep default | awk '{ print $5 }'}.chomp
elsif OS.mac?
return %x{mac/mac-bridge.sh}.chomp
end
end
# Helper method to get the machine ID of a node.
# This will only be present if the node has been
# created in VirtualBox.
def get_machine_id(vm_name)
machine_id_filepath = ".vagrant/machines/#{vm_name}/virtualbox/id"
if not File.exist? machine_id_filepath
return nil
else
return File.read(machine_id_filepath)
end
end
# Helper method to determine whether all nodes are up
def all_nodes_up()
if get_machine_id("controlplane").nil?
return false
end
(1..NUM_WORKER_NODES).each do |i|
if get_machine_id("node0#{i}").nil?
return false
end
end
return true
end
# Sets up hosts file and DNS
def setup_dns(node)
# Set up /etc/hosts
node.vm.provision "setup-hosts", :type => "shell", :path => "ubuntu/vagrant/setup-hosts.sh" do |s|
s.args = [IP_NW, BUILD_MODE, NUM_WORKER_NODES, MASTER_IP_START, NODE_IP_START]
end
# Set up DNS resolution
node.vm.provision "setup-dns", type: "shell", :path => "ubuntu/update-dns.sh"
end
# Runs provisioning steps that are required by masters and workers
def provision_kubernetes_node(node)
# Set up DNS
setup_dns node
# Set up ssh
node.vm.provision "setup-ssh", :type => "shell", :path => "ubuntu/ssh.sh"
end
# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure("2") do |config|
# The most common configuration options are documented and commented below.
# For a complete reference, please see the online documentation at
# https://docs.vagrantup.com.
# Every Vagrant development environment requires a box. You can search for
# boxes at https://vagrantcloud.com/search.
# config.vm.box = "base"
config.vm.box = "ubuntu/jammy64"
config.vm.boot_timeout = 900
# Disable automatic box update checking. If you disable this, then
# boxes will only be checked for updates when the user runs
# `vagrant box outdated`. This is not recommended.
config.vm.box_check_update = false
# Provision Master Nodes
config.vm.define "controlplane" do |node|
# Name shown in the GUI
node.vm.provider "virtualbox" do |vb|
vb.name = "controlplane"
vb.memory = 2048
vb.cpus = 2
end
node.vm.hostname = "controlplane"
if BUILD_MODE == "BRIDGE"
adapter = ""
node.vm.network :public_network, bridge: get_bridge_adapter()
else
node.vm.network :private_network, ip: IP_NW + ".#{MASTER_IP_START}"
node.vm.network "forwarded_port", guest: 22, host: "#{2710}"
end
provision_kubernetes_node node
# Install (opinionated) configs for vim and tmux on master-1. These used by the author for CKA exam.
node.vm.provision "file", source: "./ubuntu/tmux.conf", destination: "$HOME/.tmux.conf"
node.vm.provision "file", source: "./ubuntu/vimrc", destination: "$HOME/.vimrc"
end
# Provision Worker Nodes
(1..NUM_WORKER_NODES).each do |i|
config.vm.define "node0#{i}" do |node|
node.vm.provider "virtualbox" do |vb|
vb.name = "node0#{i}"
vb.memory = 1024
vb.cpus = 1
end
node.vm.hostname = "node0#{i}"
if BUILD_MODE == "BRIDGE"
node.vm.network :public_network, bridge: get_bridge_adapter()
else
node.vm.network :private_network, ip: IP_NW + ".#{NODE_IP_START + i}"
node.vm.network "forwarded_port", guest: 22, host: "#{2720 + i}"
end
provision_kubernetes_node node
end
end
if BUILD_MODE == "BRIDGE"
# Trigger that fires after each VM starts.
# Does nothing until all the VMs have started, at which point it
# gathers the IP addresses assigned to the bridge interfaces by DHCP
# and pushes a hosts file to each node with these IPs.
config.trigger.after :up do |trigger|
trigger.name = "Post provisioner"
trigger.ignore = [:destroy, :halt]
trigger.ruby do |env, machine|
if all_nodes_up()
puts " Gathering IP addresses of nodes..."
nodes = ["controlplane"]
ips = []
(1..NUM_WORKER_NODES).each do |i|
nodes.push("node0#{i}")
end
nodes.each do |n|
ips.push(%x{vagrant ssh #{n} -c 'public-ip'}.chomp)
end
hosts = ""
ips.each_with_index do |ip, i|
hosts << ip << " " << nodes[i] << "\n"
end
puts " Setting /etc/hosts on nodes..."
File.open("hosts.tmp", "w") { |file| file.write(hosts) }
nodes.each do |node|
system("vagrant upload hosts.tmp /tmp/hosts.tmp #{node}")
system("vagrant ssh #{node} -c 'cat /tmp/hosts.tmp | sudo tee -a /etc/hosts'")
end
File.delete("hosts.tmp")
puts <<~EOF
VM build complete!
Use either of the following to access any NodePort services you create from your browser
replacing "port_number" with the number of your NodePort.
EOF
(1..NUM_WORKER_NODES).each do |i|
puts " http://#{ips[i]}:port_number"
end
puts ""
else
puts " Nothing to do here"
end
end
end
end
end
================================================
FILE: kubeadm-clusters/virtualbox/docs/01-prerequisites.md
================================================
# Prerequisites
If you have an M-series (Apple Silicon) Mac, you cannot run VirtualBox. Please instead see our [Apple Silicon](../../apple-silicon/) guide.
## VM Hardware Requirements
* 8 GB of RAM (16 preferred)
* 8-core/4-core hyperthreaded or better CPU, e.g. Core-i7/Core-i9 (will be slow otherwise)
* 50 GB Disk space
## git
Required to download the repo. It is normally pre-installed on Mac, but not on Windows. If you need to install it, see [here](https://git-scm.com/download).
## VirtualBox
Download and Install [VirtualBox](https://www.virtualbox.org/wiki/Downloads) on any one of the supported platforms:
- Windows hosts
- MacOS hosts x86 only. For Apple Silicon (M-series processors), see [here](../../apple-silicon/).
- Linux distributions
This lab was last tested with VirtualBox 7.0.12, though newer versions should be ok.
## Vagrant
Once VirtualBox is installed you may choose to deploy virtual machines manually on it.
Vagrant provides an easier way to deploy multiple virtual machines on VirtualBox more consistently.
Download and Install [Vagrant](https://www.vagrantup.com/) on your platform.
This lab was last tested with Vagrant 2.3.7, though newer versions should be ok.
## Lab Defaults
The labs have been configured with the following networking defaults. If you change any of these after you have deployed any of the lab, you'll need to completely reset it and start again from the beginning:
```bash
vagrant destroy -f
vagrant up
```
If you do change any of these, **please consider that a personal preference and don't submit a PR for it**.
### Virtual Machine Network
Due to how VirtualBox/Vagrant works, the networking for each VM requires two network adapters; one NAT (`enp0s3`) to communicate with the outside world, and one internal (`enp0s8`) which is attached to the VirtualBox network mentioned above. By default, Kubernetes components will connect to the default network adapter - the NAT one, which is *not* what we want, therefore we have pre-set an environment variable `PRIMARY_IP` on all VMs which is the IP address that Kubernetes components should be using. In the coming labs you will see this environment variable being used to ensure Kubernetes components bind to the correct network interface.
`PRIMARY_IP` is defined as the IP address of the network interface on the node that is connected to the network having the default gateway, and is the interface that a node will use to talk to the other nodes. For those interested, this variable is assigned the result of the following command
```bash
ip route | grep default | awk '{ print $9 }'
```
#### Bridge Networking
The default configuration in this lab is to bring the VMs up on bridged interfaces. What this means is that your Kubernetes nodes will appear as additional machines on your local network, their IP addresses being provided dynamically by your broadband router. This facilitates the use of your browser to connect to any NodePort services you deploy.
Should you have issues deploying bridge networking, please raise a [bug report](https://github.com/kodekloudhub/certified-kubernetes-administrator-course/issues) and include all details including the output of `vagrant up`.
Then retry the lab in NAT mode. How to do this is covered in the [next section](./02-compute-resources.md).
#### NAT Networking
In NAT configuration, the network on which the VMs run is isolated from your broadband router's network by a NAT gateway managed by the hypervisor. This means that VMs can see out (and connect to Internet), but you can't see in (i.e. use browser to connect to NodePorts) without setting up individual port forwarding rules for every NodePort using the VirtualBox UI.
The network used by the Virtual Box virtual machines is `192.168.56.0/24`.
To change this, edit the [Vagrantfile](../Vagrantfile) in your cloned copy (do not edit directly in github), and set the new value for the network prefix at line 9. This should not overlap any of the other network settings.
Note that you do not need to edit any of the other scripts to make the above change. It is all managed by shell variable computations based on the assigned VM IP addresses and the values in the hosts file (also computed).
It is *recommended* that you leave the pod and service networks as the defaults. If you change them then you will also need to edit the Weave networking manifests to accommodate your change.
If you do decide to change any of these, please treat as personal preference and do not raise a pull request.
### Pod Network
The network used to assign IP addresses to pods is `10.244.0.0/16`.
To change this, open all the `.md` files in the [docs](../docs/) directory in your favourite IDE and do a global replace on
`POD_CIDR=10.244.0.0/16`
with the new CIDR range. This should not overlap any of the other network settings.
### Service Network
The network used to assign IP addresses to Cluster IP services is `10.96.0.0/16`.
To change this, open all the `.md` files in the [docs](../docs/) directory in your favourite IDE and do a global replace on
`SERVICE_CIDR=10.96.0.0/16`
with the new CIDR range. This should not overlap any of the other network settings.
## Running Commands in Parallel with tmux
If you are running this tutorial on an x86 Mac, you can instead use iterm2 to achive this. See the iterm2 setup in the [Apple Silicon guide](../../apple-silicon/docs/01-prerequisites.md#running-commands-in-parallel-with-iterm2).
[tmux](https://github.com/tmux/tmux/wiki) can be used to run the same commands on multiple compute instances at the same time. Labs in this tutorial may require running the same commands across multiple compute instances. In those cases you may consider using tmux and splitting a window into multiple panes with synchronize-panes enabled to speed up the provisioning process.
In order to use tmux, you must first connect to `controlplane` and run tmux there. From inside the tmux session you can open multiple panes and ssh to the worker nodes from these panes.
*The use of tmux is optional and not required to complete this tutorial*.
> Enable synchronize-panes by pressing `CTRL+B` followed by `"` to split the window into two panes. In each pane (selectable with mouse), ssh to the host(s) you will be working with.Next type `CTRL+X` at the prompt to begin sync. In sync mode, the dividing line between panes will be red. Everything you type or paste in one pane will be echoed in the other.
To disable synchronization type `CTRL+X` again.Note that the `CTRL-X` key binding is provided by a `.tmux.conf` loaded onto the VM by the vagrant provisioner.
To paste commands into a tmux pane, use `SHIFT-RightMouseButton`.
Next: [Compute Resources](02-compute-resources.md)
================================================
FILE: kubeadm-clusters/virtualbox/docs/02-compute-resources.md
================================================
# Provisioning Compute Resources
Note: You must have VirtualBox and Vagrant configured at this point
Open a terminal application (on Windows use PowerShell). All commands in this guide are executed from the terminal.
Download this github repository and cd into the `virtualbox` folder
```bash
git clone https://github.com/kodekloudhub/certified-kubernetes-administrator-course.git
```
CD into the virtualbox directory
```bash
cd kubeadm-clusters/virtualbox
```
Run Vagrant up to create the virtual machines.
```bash
vagrant up
```
## Bridge interface selection
Bridged networking makes the VMs appear as hosts directly on your local network. This means that you will be able to use your own browser to connect to any NodePort services you create in the cluster.
If your workstation has more than one network interface capable of creating a bridge, Vagrant may stop and ask you which one to use if it cannot determine the best interface itself. Now it *should* work this out and not have to ask you, however if it does not, then the "best" interface is the one used to connect to your broadband router. On laptops, this would normally be the Wi-Fi adapter which should be easliy identifiable in the list. The example below is from a Windows desktop computer with a wired network adapter.
Which of the two choices do you think is correct?
Reveal
> `Intel(R) Ethernet Connection (2) I219-V`
Why? Because
1. Ethernet is the term often given to wired network connections.
2. The other one is Hyper-V which is internal and used for native running of VMs (could indeed be used instead of VirtualBox, but that's another story).
```text
==> controlplane: Available bridged network interfaces:
1) Intel(R) Ethernet Connection (2) I219-V
2) Hyper-V Virtual Ethernet Adapter
==> controlplane: When choosing an interface, it is usually the one that is
==> controlplane: being used to connect to the internet.
==> controlplane:
controlplane: Which interface should the network bridge to?
```
At the end of the deployment, it will tell you how to access NodePort services from your browser once you have configured Kubernetes. Make a note of this.
If you encountered issues starting the VMs, you can try NAT mode. Note that in NAT mode you will not be able to connect to your NodePort services using your browser without setting up port forwarding rules in VirtualBox UI.
1. Run
```
vagrant destroy -f
```
1. Edit `vagrantfile` and change `BUILD_MODE = "BRIDGE"` to `BUILD_MODE = "NAT"` at line 10.
## SSH to the nodes
There are two ways to SSH into the nodes:
### 1. SSH using Vagrant
From the directory you ran the `vagrant up` command, run `vagrant ssh ` for example `vagrant ssh controlplane`.
This is the easiest way as it requires no configuration.
### 2. SSH Using SSH Client Tools
Use your favourite SSH Terminal tool (PuTTY/MobaXTerm etc.).
Use the above IP addresses. Username and password based SSH is disabled by default.
Vagrant generates a private key for each of these VMs. It is placed under the .vagrant folder (in the directory you ran the `vagrant up` command from) at the below path for each VM:
**Private Key Path:** `.vagrant/machines//virtualbox/private_key`
**Username/Password:** `vagrant/vagrant`
## Verify Environment
- Ensure all VMs are up
- Ensure VMs are assigned the above IP addresses
- Ensure you can SSH into these VMs using the IP and private keys, or `vagrant ssh`
- Ensure the VMs can ping each other
## Troubleshooting Tips
### Failed Provisioning
If any of the VMs failed to provision, or is not configured correct, delete the VM using the command:
```bash
vagrant destroy
```
Then re-provision. Only the missing VMs will be re-provisioned
```bash
vagrant up
```
Sometimes the delete does not delete the folder created for the VM and throws an error similar to this:
VirtualBox error:
VBoxManage.exe: error: Could not rename the directory 'D:\VirtualBox VMs\ubuntu-bionic-18.04-cloudimg-20190122_1552891552601_76806' to 'D:\VirtualBox VMs\kubernetes-ha-worker-2' to save the settings file (VERR_ALREADY_EXISTS)
VBoxManage.exe: error: Details: code E_FAIL (0x80004005), component SessionMachine, interface IMachine, callee IUnknown
VBoxManage.exe: error: Context: "SaveSettings()" at line 3105 of file VBoxManageModifyVM.cpp
In such cases delete the VM, then delete the VM folder and then re-provision, e.g.
```bash
vagrant destroy node02
rmdir "\node02
vagrant up
```
### Provisioner gets stuck
This will most likely happen at "Waiting for machine to reboot"
1. Hit `CTRL+C`
1. Kill any running `ruby` process, or Vagrant will complain.
1. Destroy the VM that got stuck: `vagrant destroy `
1. Re-provision. It will pick up where it left off: `vagrant up`
# Pausing the Environment
You do not need to complete the entire lab in one session. You may shut down and resume the environment as follows, if you need to power off your computer.
To shut down. This will gracefully shut down all the VMs in the reverse order to which they were started:
```
vagrant halt
```
To power on again:
```
vagrant up
```
# Deleting the Virtual Machines
When you have finished with your cluster and want to reclaim the resources, perform the following steps
1. Exit from all your VM sessions
1. Run the following
```
vagrant destroy -f
```
Next: [Connectivity](./03-connectivity.md)
Prev: [Prerequisites](./01-prerequisites.md)
================================================
FILE: kubeadm-clusters/virtualbox/docs/03-connectivity.md
================================================
# Connectivity
First identify a system from where you will perform administrative tasks, such as creating certificates, kubeconfig files and distributing them to the different VMs.
If you are on a Linux laptop, then your laptop could be this system. In my case I chose the `controlplane` node to perform administrative tasks. Whichever system you chose make sure that system is able to access all the provisioned VMs through SSH to copy files over.
## Access all VMs
Here we create an SSH key pair for the `vagrant` user who we are logged in as. We will copy the public key of this pair to both workers to permit us to use password-less SSH (and SCP) to get from `controlplane` to these other nodes in the context of the `vagrant` user which exists on all nodes.
Generate Key Pair on `controlplane` node
[//]: # (host:controlplane)
```bash
ssh-keygen
```
Leave all settings to default (just press ENTER at any questions).
Copy the key to the other hosts. For this step please enter `vagrant` where a password is requested.
The option `-o StrictHostKeyChecking=no` tells it not to ask if you want to connect to a previously unknown host. Not best practice in the real world, but speeds things up here.
```bash
ssh-copy-id -o StrictHostKeyChecking=no vagrant@node01
```
```bash
ssh-copy-id -o StrictHostKeyChecking=no vagrant@node02
```
For each host, the output should be similar to this. If it is not, then you may have entered an incorrect password. Retry the step.
```
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'vagrant@node01'"
and check to make sure that only the key(s) you wanted were added.
```
Next: [Node Setup](../../generic/04-node-setup.md)
Prev: [Compute Resources](02-compute-resources.md)
================================================
FILE: kubeadm-clusters/virtualbox/mac/mac-bridge.sh
================================================
#!/bin/bash
iface=$(netstat -rn -f inet | grep '^default' | sort | head -1 | awk '{print $4}')
interface=$(VBoxManage list bridgedifs | awk '/^Name:/ { print }' | sed -e 's/^Name:[ ]*\(.*\)/\1/' | grep $iface)
echo -n $interface
================================================
FILE: kubeadm-clusters/virtualbox/tools/lab-script-generator.py
================================================
#!/usr/bin/env python3
#
# Generate a set of scripts from the documentation by reading markdown comments to determine target hosts
# and extracting all scripts fenced by ```bash
#
#
# Point it to the docs directory containing the lab documents
# It will create shell scripts in docs/../quick-steps
# THe scripts are ordered by lab number, then a, b, c where a lab requires you to use more than one host,
# and the filename indicates which hosts to run
#
# Hosts and other things are determined from markdown comments.
# For documents containing no markdown comments, scripts are not generated.
import re
import os
import glob
import codecs
import argparse
from enum import Enum
from sys import exit
class State(Enum):
NONE = 0
SCRIPT = 1
parser = argparse.ArgumentParser(description="Extract scripts from markdown")
parser.add_argument("--path", '-p', required=True, help='Path to markdown docs')
args = parser.parse_args()
docs_path = os.path.abspath(args.path)
if not os.path.isdir(docs_path):
print (f'Invalid path: {docs_path}')
exit(1)
qs_path = os.path.abspath(os.path.join(docs_path, '../quick-steps'))
if not os.path.isdir(qs_path):
os.makedirs(qs_path)
newline = chr(10) # In case running on Windows (plus writing files as binary to not convert to \r\n)
file_number_rx = re.compile(r'^(?P\d+)')
comment_rx = re.compile(r'^\[//\]:\s\#\s\((?P\w+):(?P.*)\)\s*$')
choice_rx = re.compile(r'^\s*-+\s+OR\s+-+')
script_begin = '```bash'
script_end = '```'
script_open = ('{' + newline).encode('utf-8')
script_close = '\n}'.encode('utf-8')
current_host = None
file_nos = []
def write_script(filename: str, script: list):
path = os.path.join(qs_path, filename)
with open(path, "wb") as f:
f.write(script_open)
f.write(newline.join(script).encode('utf-8'))
f.write(script_close)
print(f'-> {path}')
output_file_no = 1
script = []
output_file = None
for doc in glob.glob(os.path.join(docs_path, '*.md')):
print(doc)
state = State.NONE
ignore_next_script = False
m = file_number_rx.search(os.path.basename(doc))
if not m:
continue
file_no = m['number']
if int(file_no) < 3:
continue
file_nos.append(file_no)
section = 0
script.extend([
"##################################################",
"#",
f"# {os.path.basename(doc)}",
"#",
"##################################################",
""
])
with codecs.open(doc, "r", encoding='utf-8') as f:
for line in f.readlines():
line = line.rstrip()
if state == State.NONE:
m = comment_rx.search(line)
if m:
token = m['token']
value = m['value']
if token == 'host':
if script and current_host and current_host != value:
#fns = file_no if len(file_nos) < 2 else '-'.join(file_nos[:-1])
script.append('set +e')
output_file = os.path.join(qs_path, f'{output_file_no}-{current_host}.sh')
write_script(output_file, script)
output_file_no += 1
script = [
"##################################################",
"#",
f"# {os.path.basename(doc)}",
"#",
"##################################################",
""
]
file_nos = [file_no]
output_file = os.path.join(qs_path, f'{file_no}{chr(97 + section)}-{value}.sh')
section += 1
current_host = value
elif token == 'sleep':
script.extend([
f'echo "Sleeping {value}s"',
f'sleep {value}',
newline
])
elif token == 'command':
script.extend([
value,
newline
])
elif token == 'comment':
script.extend([
'#######################################################################',
'#',
f'# {value}',
'#',
'#######################################################################',
newline
])
elif line.strip() == script_begin:
state = State.SCRIPT
elif choice_rx.match(line):
ignore_next_script = True
elif state == State.SCRIPT:
if line.strip() == script_end:
state = State.NONE
script.append(newline)
ignore_next_script = False
# elif line.startswith('source') or line.startswith('export'):
# script.append('}')
# script.append(line)
# script.append('{')
elif not (ignore_next_script or line == '{' or line == '}'):
script.append(line.strip())
if script:
# fns = '-'.join(file_nos[1:])
output_file = os.path.join(qs_path, f'{output_file_no}-{current_host}.sh')
write_script(output_file, script)
================================================
FILE: kubeadm-clusters/virtualbox/ubuntu/cert_verify.sh
================================================
#!/bin/bash
set -e
#set -x
# Green & Red marking for Success and Failed messages
SUCCESS='\033[0;32m'
FAILED='\033[0;31;1m'
NC='\033[0m'
# IP addresses
PRIMARY_IP=$(ip addr show enp0s8 | grep "inet " | awk '{print $2}' | cut -d / -f 1)
MASTER_1=$(dig +short master-1)
MASTER_2=$(dig +short master-2)
WORKER_1=$(dig +short worker-1)
WORKER_2=$(dig +short worker-2)
LOADBALANCER=$(dig +short loadbalancer)
LOCALHOST="127.0.0.1"
# All Cert Location
# ca certificate location
CACERT=ca.crt
CAKEY=ca.key
# Kube controller manager certificate location
KCMCERT=kube-controller-manager.crt
KCMKEY=kube-controller-manager.key
# Kube proxy certificate location
KPCERT=kube-proxy.crt
KPKEY=kube-proxy.key
# Kube scheduler certificate location
KSCERT=kube-scheduler.crt
KSKEY=kube-scheduler.key
# Kube api certificate location
APICERT=kube-apiserver.crt
APIKEY=kube-apiserver.key
# ETCD certificate location
ETCDCERT=etcd-server.crt
ETCDKEY=etcd-server.key
# Service account certificate location
SACERT=service-account.crt
SAKEY=service-account.key
# All kubeconfig locations
# kubeproxy.kubeconfig location
KPKUBECONFIG=kube-proxy.kubeconfig
# kube-controller-manager.kubeconfig location
KCMKUBECONFIG=kube-controller-manager.kubeconfig
# kube-scheduler.kubeconfig location
KSKUBECONFIG=kube-scheduler.kubeconfig
# admin.kubeconfig location
ADMINKUBECONFIG=admin.kubeconfig
# All systemd service locations
# etcd systemd service
SYSTEMD_ETCD_FILE=/etc/systemd/system/etcd.service
# kub-api systemd service
SYSTEMD_API_FILE=/etc/systemd/system/kube-apiserver.service
# kube-controller-manager systemd service
SYSTEMD_KCM_FILE=/etc/systemd/system/kube-controller-manager.service
# kube-scheduler systemd service
SYSTEMD_KS_FILE=/etc/systemd/system/kube-scheduler.service
### WORKER NODES ###
# Worker-1 cert details
WORKER_1_CERT=/var/lib/kubelet/worker-1.crt
WORKER_1_KEY=/var/lib/kubelet/worker-1.key
# Worker-1 kubeconfig location
WORKER_1_KUBECONFIG=/var/lib/kubelet/kubeconfig
# Worker-1 kubelet config location
WORKER_1_KUBELET=/var/lib/kubelet/kubelet-config.yaml
# Systemd worker-1 kubelet location
SYSTEMD_WORKER_1_KUBELET=/etc/systemd/system/kubelet.service
# kube-proxy worker-1 location
WORKER_1_KP_KUBECONFIG=/var/lib/kube-proxy/kubeconfig
SYSTEMD_WORKER_1_KP=/etc/systemd/system/kube-proxy.service
# Function - Master node #
check_cert_and_key()
{
local name=$1
local subject=$2
local issuer=$3
local nokey=
local cert="${CERT_LOCATION}/$1.crt"
local key="${CERT_LOCATION}/$1.key"
if [ -z $cert -o -z $key ]
then
printf "${FAILED}cert and/or key not present in ${CERT_LOCATION}. Perhaps you missed a copy step\n${NC}"
exit 1
elif [ -f $cert -a -f $key ]
then
printf "${NC}${name} cert and key found, verifying the authenticity\n"
CERT_SUBJECT=$(sudo openssl x509 -in $cert -text | grep "Subject: CN"| tr -d " ")
CERT_ISSUER=$(sudo openssl x509 -in $cert -text | grep "Issuer: CN"| tr -d " ")
CERT_MD5=$(sudo openssl x509 -noout -modulus -in $cert | openssl md5| awk '{print $2}')
KEY_MD5=$(sudo openssl rsa -noout -modulus -in $key | openssl md5| awk '{print $2}')
if [ $CERT_SUBJECT == "${subject}" ] && [ $CERT_ISSUER == "${issuer}" ] && [ $CERT_MD5 == $KEY_MD5 ]
then
printf "${SUCCESS}${name} cert and key are correct\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the ${name} certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#certificate-authority\n${NC}"
exit 1
fi
else
printf "${FAILED}${cert} / ${key} is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#certificate-authority\n"
echo "These should be in /var/lib/kubernetes/pki (most certs), /etc/etcd (eccd server certs) or /var/lib/kubelet (kubelet certs)${NC}"
exit 1
fi
}
check_cert_only()
{
local name=$1
local subject=$2
local issuer=$3
local cert="${CERT_LOCATION}/$1.crt"
# Worker-2 auto cert is a .pem
[ -f "${CERT_LOCATION}/$1.pem" ] && cert="${CERT_LOCATION}/$1.pem"
if [ -z $cert ]
then
printf "${FAILED}cert not present in ${CERT_LOCATION}. Perhaps you missed a copy step\n${NC}"
exit 1
elif [ -f $cert ]
then
printf "${NC}${name} cert found, verifying the authenticity\n"
CERT_SUBJECT=$(sudo openssl x509 -in $cert -text | grep "Subject: "| tr -d " ")
CERT_ISSUER=$(sudo openssl x509 -in $cert -text | grep "Issuer: CN"| tr -d " ")
CERT_MD5=$(sudo openssl x509 -noout -modulus -in $cert | openssl md5| awk '{print $2}')
if [ $CERT_SUBJECT == "${subject}" ] && [ $CERT_ISSUER == "${issuer}" ]
then
printf "${SUCCESS}${name} cert is correct\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the ${name} certificate, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#certificate-authority\n${NC}"
exit 1
fi
else
printf "${FAILED}${cert} missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md#certificate-authority\n${NC}"
echo "These should be in ${CERT_LOCATION}${NC}"
exit 1
fi
}
check_cert_adminkubeconfig()
{
if [ -z $ADMINKUBECONFIG ]
then
printf "${FAILED}please specify admin kubeconfig location\n${NC}"
exit 1
elif [ -f $ADMINKUBECONFIG ]
then
printf "${NC}admin kubeconfig file found, verifying the authenticity\n"
ADMINKUBECONFIG_SUBJECT=$(cat $ADMINKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | sudo openssl x509 -text | grep "Subject: CN" | tr -d " ")
ADMINKUBECONFIG_ISSUER=$(cat $ADMINKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | sudo openssl x509 -text | grep "Issuer: CN" | tr -d " ")
ADMINKUBECONFIG_CERT_MD5=$(cat $ADMINKUBECONFIG | grep "client-certificate-data:" | awk '{print $2}' | base64 --decode | sudo openssl x509 -noout | openssl md5 | awk '{print $2}')
ADMINKUBECONFIG_KEY_MD5=$(cat $ADMINKUBECONFIG | grep "client-key-data" | awk '{print $2}' | base64 --decode | openssl rsa -noout | openssl md5 | awk '{print $2}')
ADMINKUBECONFIG_SERVER=$(cat $ADMINKUBECONFIG | grep "server:"| awk '{print $2}')
if [ $ADMINKUBECONFIG_SUBJECT == "Subject:CN=admin,O=system:masters" ] && [ $ADMINKUBECONFIG_ISSUER == "Issuer:CN=KUBERNETES-CA,O=Kubernetes" ] && [ $ADMINKUBECONFIG_CERT_MD5 == $ADMINKUBECONFIG_KEY_MD5 ] && [ $ADMINKUBECONFIG_SERVER == "https://127.0.0.1:6443" ]
then
printf "${SUCCESS}admin kubeconfig cert and key are correct\n"
else
printf "${FAILED}Exiting...Found mismtach in the admin kubeconfig certificate and keys, More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/05-kubernetes-configuration-files.md#the-admin-kubernetes-configuration-file\n"
exit 1
fi
else
printf "${FAILED}admin kubeconfig file is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/05-kubernetes-configuration-files.md#the-admin-kubernetes-configuration-file\n"
exit 1
fi
}
get_kubeconfig_cert_path()
{
local kubeconfig=$1
local cert_field=$2
sudo cat $kubeconfig | grep cert_field | awk '{print $2}'
}
check_kubeconfig()
{
local name=$1
local location=$2
local apiserver=$3
local kubeconfig="${location}/${name}.kubeconfig"
echo "Checking $kubeconfig"
check_kubeconfig_exists $name $location
ca=$(get_kubeconfig_cert_path $kubeconfig "certificate-authority")
cert=$(get_kubeconfig_cert_path $kubeconfig "client-certificate")
key=$(get_kubeconfig_cert_path $kubeconfig "client-key")
server=$(sudo cat $kubeconfig | grep server | awk '{print $2}')
if [ -f "$ca"]
then
printf "${SUCCESS}Path to CA certificate is correct${NC}\n"
else
printf "${FAIL}CA certificate not found at ${ca}${NC}\n"
exit 1
fi
if [ -f "$cert"]
then
printf "${SUCCESS}Path to client certificate is correct${NC}\n"
else
printf "${FAIL}Client certificate not found at ${cert}${NC}\n"
exit 1
fi
if [ -f "$key"]
then
printf "${SUCCESS}Path to client key is correct${NC}\n"
else
printf "${FAIL}Client key not found at ${key}${NC}\n"
exit 1
fi
if [ "$apiserver" = "$server" ]
then
printf "${SUCCESS}Server URL is correct${NC}\n"
else
printf "${FAIL}Server URL ${server} is incorrect${NC}\n"
exit 1
fi
}
check_kubeconfig_exists() {
local name=$1
local location=$2
local kubeconfig="${location}/${name}.kubeconfig"
if [ -f "${kubeconfig}" ]
then
printf "${SUCCESS}${kubeconfig} found${NC}\n"
else
printf "${FAIL}${kubeconfig} not found!${NC}\n"
exit 1
fi
}
check_systemd_etcd()
{
if [ -z $ETCDCERT ] && [ -z $ETCDKEY ]
then
printf "${FAILED}please specify ETCD cert and key location, Exiting....\n${NC}"
exit 1
elif [ -f $SYSTEMD_ETCD_FILE ]
then
printf "${NC}Systemd for ETCD service found, verifying the authenticity\n"
# Systemd cert and key file details
ETCD_CA_CERT=ca.crt
CERT_FILE=$(systemctl cat etcd.service | grep "\--cert-file"| awk '{print $1}'| cut -d "=" -f2)
KEY_FILE=$(systemctl cat etcd.service | grep "\--key-file"| awk '{print $1}' | cut -d "=" -f2)
PEER_CERT_FILE=$(systemctl cat etcd.service | grep "\--peer-cert-file"| awk '{print $1}'| cut -d "=" -f2)
PEER_KEY_FILE=$(systemctl cat etcd.service | grep "\--peer-key-file"| awk '{print $1}'| cut -d "=" -f2)
TRUSTED_CA_FILE=$(systemctl cat etcd.service | grep "\--trusted-ca-file"| awk '{print $1}'| cut -d "=" -f2)
PEER_TRUSTED_CA_FILE=$(systemctl cat etcd.service | grep "\--peer-trusted-ca-file"| awk '{print $1}'| cut -d "=" -f2)
# Systemd advertise , client and peer url's
IAP_URL=$(systemctl cat etcd.service | grep "\--initial-advertise-peer-urls"| awk '{print $2}')
LP_URL=$(systemctl cat etcd.service | grep "\--listen-peer-urls"| awk '{print $2}')
LC_URL=$(systemctl cat etcd.service | grep "\--listen-client-urls"| awk '{print $2}')
AC_URL=$(systemctl cat etcd.service | grep "\--advertise-client-urls"| awk '{print $2}')
ETCD_CA_CERT=/etc/etcd/ca.crt
ETCDCERT=/etc/etcd/etcd-server.crt
ETCDKEY=/etc/etcd/etcd-server.key
if [ $CERT_FILE == $ETCDCERT ] && [ $KEY_FILE == $ETCDKEY ] && [ $PEER_CERT_FILE == $ETCDCERT ] && [ $PEER_KEY_FILE == $ETCDKEY ] && \
[ $TRUSTED_CA_FILE == $ETCD_CA_CERT ] && [ $PEER_TRUSTED_CA_FILE = $ETCD_CA_CERT ]
then
printf "${SUCCESS}ETCD certificate, ca and key files are correct under systemd service\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the ETCD certificate, ca and keys. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/07-bootstrapping-etcd.md#configure-the-etcd-server\n${NC}"
exit 1
fi
if [ $IAP_URL == "https://$PRIMARY_IP:2380" ] && [ $LP_URL == "https://$PRIMARY_IP:2380" ] && [ $LC_URL == "https://$PRIMARY_IP:2379,https://127.0.0.1:2379" ] && \
[ $AC_URL == "https://$PRIMARY_IP:2379" ]
then
printf "${SUCCESS}ETCD initial-advertise-peer-urls, listen-peer-urls, listen-client-urls, advertise-client-urls are correct\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the ETCD initial-advertise-peer-urls / listen-peer-urls / listen-client-urls / advertise-client-urls. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/07-bootstrapping-etcd.md#configure-the-etcd-server\n${NC}"
exit 1
fi
else
printf "${FAILED}etcd-server.crt / etcd-server.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/07-bootstrapping-etcd.md#configure-the-etcd-server\n${NC}"
exit 1
fi
}
check_systemd_api()
{
if [ -z $APICERT ] && [ -z $APIKEY ]
then
printf "${FAILED}please specify kube-api cert and key location, Exiting....\n${NC}"
exit 1
elif [ -f $SYSTEMD_API_FILE ]
then
printf "Systemd for kube-api service found, verifying the authenticity\n"
ADVERTISE_ADDRESS=$(systemctl cat kube-apiserver.service | grep "\--advertise-address" | awk '{print $1}' | cut -d "=" -f2)
CLIENT_CA_FILE=$(systemctl cat kube-apiserver.service | grep "\--client-ca-file" | awk '{print $1}' | cut -d "=" -f2)
ETCD_CA_FILE=$(systemctl cat kube-apiserver.service | grep "\--etcd-cafile" | awk '{print $1}' | cut -d "=" -f2)
ETCD_CERT_FILE=$(systemctl cat kube-apiserver.service | grep "\--etcd-certfile" | awk '{print $1}' | cut -d "=" -f2)
ETCD_KEY_FILE=$(systemctl cat kube-apiserver.service | grep "\--etcd-keyfile" | awk '{print $1}' | cut -d "=" -f2)
KUBELET_CERTIFICATE_AUTHORITY=$(systemctl cat kube-apiserver.service | grep "\--kubelet-certificate-authority" | awk '{print $1}' | cut -d "=" -f2)
KUBELET_CLIENT_CERTIFICATE=$(systemctl cat kube-apiserver.service | grep "\--kubelet-client-certificate" | awk '{print $1}' | cut -d "=" -f2)
KUBELET_CLIENT_KEY=$(systemctl cat kube-apiserver.service | grep "\--kubelet-client-key" | awk '{print $1}' | cut -d "=" -f2)
SERVICE_ACCOUNT_KEY_FILE=$(systemctl cat kube-apiserver.service | grep "\--service-account-key-file" | awk '{print $1}' | cut -d "=" -f2)
TLS_CERT_FILE=$(systemctl cat kube-apiserver.service | grep "\--tls-cert-file" | awk '{print $1}' | cut -d "=" -f2)
TLS_PRIVATE_KEY_FILE=$(systemctl cat kube-apiserver.service | grep "\--tls-private-key-file" | awk '{print $1}' | cut -d "=" -f2)
PKI=/var/lib/kubernetes/pki
CACERT="${PKI}/ca.crt"
APICERT="${PKI}/kube-apiserver.crt"
APIKEY="${PKI}/kube-apiserver.key"
SACERT="${PKI}/service-account.crt"
KCCERT="${PKI}/apiserver-kubelet-client.crt"
KCKEY="${PKI}/apiserver-kubelet-client.key"
if [ $ADVERTISE_ADDRESS == $PRIMARY_IP ] && [ $CLIENT_CA_FILE == $CACERT ] && [ $ETCD_CA_FILE == $CACERT ] && \
[ $ETCD_CERT_FILE == "${PKI}/etcd-server.crt" ] && [ $ETCD_KEY_FILE == "${PKI}/etcd-server.key" ] && \
[ $KUBELET_CERTIFICATE_AUTHORITY == $CACERT ] && [ $KUBELET_CLIENT_CERTIFICATE == $KCCERT ] && [ $KUBELET_CLIENT_KEY == $KCKEY ] && \
[ $SERVICE_ACCOUNT_KEY_FILE == $SACERT ] && [ $TLS_CERT_FILE == $APICERT ] && [ $TLS_PRIVATE_KEY_FILE == $APIKEY ]
then
printf "${SUCCESS}kube-apiserver advertise-address/ client-ca-file/ etcd-cafile/ etcd-certfile/ etcd-keyfile/ kubelet-certificate-authority/ kubelet-client-certificate/ kubelet-client-key/ service-account-key-file/ tls-cert-file/ tls-private-key-file are correct\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the kube-apiserver systemd file, check advertise-address/ client-ca-file/ etcd-cafile/ etcd-certfile/ etcd-keyfile/ kubelet-certificate-authority/ kubelet-client-certificate/ kubelet-client-key/ service-account-key-file/ tls-cert-file/ tls-private-key-file. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-api-server\n${NC}"
exit 1
fi
else
printf "${FAILED}kube-apiserver.crt / kube-apiserver.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-api-server\n${NC}"
exit 1
fi
}
check_systemd_kcm()
{
KCMCERT=/var/lib/kubernetes/pki/kube-controller-manager.crt
KCMKEY=/var/lib/kubernetes/pki/kube-controller-manager.key
CACERT=/var/lib/kubernetes/pki/ca.crt
CAKEY=/var/lib/kubernetes/pki/ca.key
SAKEY=/var/lib/kubernetes/pki/service-account.key
KCMKUBECONFIG=/var/lib/kubernetes/kube-controller-manager.kubeconfig
if [ -z $KCMCERT ] && [ -z $KCMKEY ]
then
printf "${FAILED}please specify cert and key location\n${NC}"
exit 1
elif [ -f $SYSTEMD_KCM_FILE ]
then
printf "Systemd for kube-controller-manager service found, verifying the authenticity\n"
CLUSTER_SIGNING_CERT_FILE=$(systemctl cat kube-controller-manager.service | grep "\--cluster-signing-cert-file" | awk '{print $1}' | cut -d "=" -f2)
CLUSTER_SIGNING_KEY_FILE=$(systemctl cat kube-controller-manager.service | grep "\--cluster-signing-key-file" | awk '{print $1}' | cut -d "=" -f2)
KUBECONFIG=$(systemctl cat kube-controller-manager.service | grep "\--kubeconfig" | awk '{print $1}' | cut -d "=" -f2)
ROOT_CA_FILE=$(systemctl cat kube-controller-manager.service | grep "\--root-ca-file" | awk '{print $1}' | cut -d "=" -f2)
SERVICE_ACCOUNT_PRIVATE_KEY_FILE=$(systemctl cat kube-controller-manager.service | grep "\--service-account-private-key-file" | awk '{print $1}' | cut -d "=" -f2)
if [ $CLUSTER_SIGNING_CERT_FILE == $CACERT ] && [ $CLUSTER_SIGNING_KEY_FILE == $CAKEY ] && [ $KUBECONFIG == $KCMKUBECONFIG ] && \
[ $ROOT_CA_FILE == $CACERT ] && [ $SERVICE_ACCOUNT_PRIVATE_KEY_FILE == $SAKEY ]
then
printf "${SUCCESS}kube-controller-manager cluster-signing-cert-file, cluster-signing-key-file, kubeconfig, root-ca-file, service-account-private-key-file are correct\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the kube-controller-manager cluster-signing-cert-file, cluster-signing-key-file, kubeconfig, root-ca-file, service-account-private-key-file. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-controller-manager\n${NC}"
exit 1
fi
else
printf "${FAILED}kube-controller-manager.crt / kube-controller-manager.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-controller-manager\n${NC}"
exit 1
fi
}
check_systemd_ks()
{
KSCERT=/var/lib/kubernetes/pki/kube-scheduler.crt
KSKEY=/var/lib/kubernetes/pki/kube-scheduler.key
KSKUBECONFIG=/var/lib/kubernetes/kube-scheduler.kubeconfig
if [ -z $KSCERT ] && [ -z $KSKEY ]
then
printf "${FAILED}please specify cert and key location\n${NC}"
exit 1
elif [ -f $SYSTEMD_KS_FILE ]
then
printf "Systemd for kube-scheduler service found, verifying the authenticity\n"
KUBECONFIG=$(systemctl cat kube-scheduler.service | grep "\--kubeconfig"| awk '{print $1}'| cut -d "=" -f2)
if [ $KUBECONFIG == $KSKUBECONFIG ]
then
printf "${SUCCESS}kube-scheduler --kubeconfig is correct\n${NC}"
else
printf "${FAILED}Exiting...Found mismtach in the kube-scheduler --kubeconfig. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-scheduler\n${NC}"
exit 1
fi
else
printf "${FAILED}kube-scheduler.crt / kube-scheduler.key is missing. More details: https://github.com/mmumshad/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md#configure-the-kubernetes-scheduler\n${NC}"
exit 1
fi
}
# END OF Function - Master node #
echo "This script will validate the certificates in master as well as worker-1 nodes. Before proceeding, make sure you ssh into the respective node [ Master or Worker-1 ] for certificate validation"
echo
echo " 1. Verify certificates on Master Nodes after step 4"
echo " 2. Verify kubeconfigs on Master Nodes after step 5"
echo " 3. Verify kubeconfigs and PKI on Master Nodes after step 8"
echo " 4. Verify kubeconfigs and PKI on worker-1 Node after step 10"
echo " 5. Verify kubeconfigs and PKI on worker-2 Node after step 11"
echo
echo -n "Please select one of the above options: "
read value
HOST=$(hostname -s)
CERT_ISSUER="Issuer:CN=KUBERNETES-CA,O=Kubernetes"
SUBJ_CA="Subject:CN=KUBERNETES-CA,O=Kubernetes"
SUBJ_ADMIN="Subject:CN=admin,O=system:masters"
SUBJ_KCM="Subject:CN=system:kube-controller-manager,O=system:kube-controller-manager"
SUBJ_KP="Subject:CN=system:kube-proxy,O=system:node-proxier"
SUBJ_KS="Subject:CN=system:kube-scheduler,O=system:kube-scheduler"
SUBJ_API="Subject:CN=kube-apiserver,O=Kubernetes"
SUBJ_SA="Subject:CN=service-accounts,O=Kubernetes"
SUBJ_ETCD="Subject:CN=etcd-server,O=Kubernetes"
SUBJ_APIKC="Subject:CN=kube-apiserver-kubelet-client,O=system:masters"
case $value in
1)
if ! [ "${HOST}" = "master-1" -o "${HOST}" = "master-2" ]
then
printf "${FAILED}Must run on master-1 or master-2${NC}\n"
exit 1
fi
echo -e "The selected option is $value, proceeding the certificate verification of Master node"
CERT_LOCATION=$HOME
check_cert_and_key "ca" $SUBJ_CA $CERT_ISSUER
check_cert_and_key "kube-apiserver" $SUBJ_API $CERT_ISSUER
check_cert_and_key "kube-controller-manager" $SUBJ_KCM $CERT_ISSUER
check_cert_and_key "kube-scheduler" $SUBJ_KS $CERT_ISSUER
check_cert_and_key "service-account" $SUBJ_SA $CERT_ISSUER
check_cert_and_key "apiserver-kubelet-client" $SUBJ_APIKC $CERT_ISSUER
check_cert_and_key "etcd-server" $SUBJ_ETCD $CERT_ISSUER
if [ "${HOST}" = "master-1" ]
then
check_cert_and_key "admin" $SUBJ_ADMIN $CERT_ISSUER
check_cert_and_key "kube-proxy" $SUBJ_KP $CERT_ISSUER
fi
;;
2)
if ! [ "${HOST}" = "master-1" -o "${HOST}" = "master-2" ]
then
printf "${FAILED}Must run on master-1 or master-2${NC}\n"
exit 1
fi
check_cert_adminkubeconfig
check_kubeconfig_exists "kube-controller-manager" $HOME
check_kubeconfig_exists "kube-scheduler" $HOME
if [ "${HOST}" = "master-1" ]
then
check_kubeconfig_exists "kube-proxy" $HOME
fi
;;
3)
if ! [ "${HOST}" = "master-1" -o "${HOST}" = "master-2" ]
then
printf "${FAILED}Must run on master-1 or master-2${NC}\n"
exit 1
fi
CERT_LOCATION=/etc/etcd
check_cert_only "ca" $SUBJ_CA $CERT_ISSUER
check_cert_and_key "etcd-server" $SUBJ_ETCD $CERT_ISSUER
CERT_LOCATION=/var/lib/kubernetes/pki
check_cert_and_key "ca" $SUBJ_CA $CERT_ISSUER
check_cert_and_key "kube-apiserver" $SUBJ_API $CERT_ISSUER
check_cert_and_key "kube-controller-manager" $SUBJ_KCM $CERT_ISSUER
check_cert_and_key "kube-scheduler" $SUBJ_KS $CERT_ISSUER
check_cert_and_key "service-account" $SUBJ_SA $CERT_ISSUER
check_cert_and_key "apiserver-kubelet-client" $SUBJ_APIKC $CERT_ISSUER
check_cert_and_key "etcd-server" $SUBJ_ETCD $CERT_ISSUER
check_kubeconfig "kube-controller-manager" "/var/lib/kubernetes" "https://127.0.0.1:6443"
check_kubeconfig "kube-scheduler" "/var/lib/kubernetes" "https://127.0.0.1:6443"
check_systemd_api
check_systemd_etcd
check_systemd_kcm
check_systemd_ks
;;
4)
if ! [ "${HOST}" = "worker-1" ]
then
printf "${FAILED}Must run on worker-1${NC}\n"
exit 1
fi
CERT_LOCATION=/var/lib/kubernetes/pki
check_cert_only "ca" $SUBJ_CA $CERT_ISSUER
check_cert_and_key "kube-proxy" $SUBJ_KP $CERT_ISSUER
check_cert_and_key "worker-1" "Subject:CN=system:node:worker-1,O=system:nodes" $CERT_ISSUER
check_kubeconfig "kube-proxy" "/var/lib/kube-proxy" "https://${LOADBALANCER}:6443"
check_kubeconfig "kubelet" "/var/lib/kubelet" "https://${LOADBALANCER}:6443"
;;
5)
if ! [ "${HOST}" = "worker-2" ]
then
printf "${FAILED}Must run on worker-2${NC}\n"
exit 1
fi
CERT_LOCATION=/var/lib/kubernetes/pki
check_cert_only "ca" $SUBJ_CA $CERT_ISSUER
check_cert_and_key "kube-proxy" $SUBJ_KP $CERT_ISSUER
CERT_LOCATION=/var/lib/kubelet/pki
check_cert_only "kubelet-client-current" "Subject:O=system:nodes,CN=system:node:worker-2" $CERT_ISSUER
check_kubeconfig "kube-proxy" "/var/lib/kube-proxy" "https://${LOADBALANCER}:6443"
;;
*)
printf "${FAILED}Exiting.... Please select the valid option either 1 or 2\n${NC}"
exit 1
;;
esac
================================================
FILE: kubeadm-clusters/virtualbox/ubuntu/setup-kernel.sh
================================================
#!/bin/bash
#
# Sets up the kernel with the requirements for running Kubernetes
set -e
# Add br_netfilter kernel module
echo "br_netfilter" >> /etc/modules
# Set network tunables
cat <> /etc/sysctl.d/10-kubernetes.conf
net.bridge.bridge-nf-call-iptables=1
net.ipv4.ip_forward=1
EOF
================================================
FILE: kubeadm-clusters/virtualbox/ubuntu/ssh.sh
================================================
#!/bin/bash
# Enable password auth in sshd so we can use ssh-copy-id
sed -i 's/#PasswordAuthentication/PasswordAuthentication/' /etc/ssh/sshd_config
sed -i 's/KbdInteractiveAuthentication no/KbdInteractiveAuthentication yes/' /etc/ssh/sshd_config
systemctl restart sshd
================================================
FILE: kubeadm-clusters/virtualbox/ubuntu/tmux.conf
================================================
set -g default-shell /usr/bin/bash
set -g mouse on
bind -n C-x setw synchronize-panes
================================================
FILE: kubeadm-clusters/virtualbox/ubuntu/update-dns.sh
================================================
#!/bin/bash
# Point to Google's DNS server
sed -i -e 's/#DNS=/DNS=8.8.8.8/' /etc/systemd/resolved.conf
service systemd-resolved restart
================================================
FILE: kubeadm-clusters/virtualbox/ubuntu/vagrant/controlplane.sh
================================================
{
POD_CIDR=10.244.0.0/16
SERVICE_CIDR=10.96.0.0/16
kubeadm init --pod-network-cidr $POD_CIDR --service-cidr $SERVICE_CIDR --apiserver-advertise-address $PRIMARY_IP
kubectl --kubeconfig /etc/kubernetes/admin.conf \
apply -f "https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s-1.11.yaml"
}
================================================
FILE: kubeadm-clusters/virtualbox/ubuntu/vagrant/install-guest-additions.sh
================================================
#!/bin/bash
GUEST_ADDITION_VERSION=5.2.4
GUEST_ADDITION_ISO=VBoxGuestAdditions_${GUEST_ADDITION_VERSION}.iso
GUEST_ADDITION_MOUNT=/media/VBoxGuestAdditions
apt-get install linux-headers-$(uname -r) build-essential dkms
wget http://download.virtualbox.org/virtualbox/${GUEST_ADDITION_VERSION}/${GUEST_ADDITION_ISO}
mkdir -p ${GUEST_ADDITION_MOUNT}
mount -o loop,ro ${GUEST_ADDITION_ISO} ${GUEST_ADDITION_MOUNT}
sh ${GUEST_ADDITION_MOUNT}/VBoxLinuxAdditions.run
rm ${GUEST_ADDITION_ISO}
umount ${GUEST_ADDITION_MOUNT}
rmdir ${GUEST_ADDITION_MOUNT}
================================================
FILE: kubeadm-clusters/virtualbox/ubuntu/vagrant/node-setup.sh
================================================
{
apt-get update
apt-get install -y apt-transport-https ca-certificates curl
cat < /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
modprobe overlay
modprobe br_netfilter
cat < /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system
apt-get install -y containerd
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
systemctl restart containerd
KUBE_LATEST=$(curl -L -s https://dl.k8s.io/release/stable.txt | awk 'BEGIN { FS="." } { printf "%s.%s", $1, $2 }')
mkdir -p /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/${KUBE_LATEST}/deb/Release.key | gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/${KUBE_LATEST}/deb/ /" > /etc/apt/sources.list.d/kubernetes.list
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl
crictl config \
--set runtime-endpoint=unix:///run/containerd/containerd.sock \
--set image-endpoint=unix:///run/containerd/containerd.sock
cat < /etc/default/kubelet
KUBELET_EXTRA_ARGS='--node-ip $(ip -4 addr show enp0s8 | grep "inet" | head -1 |awk '{print $2}' | cut -d/ -f1)'
EOF
}
================================================
FILE: kubeadm-clusters/virtualbox/ubuntu/vagrant/setup-hosts.sh
================================================
#!/usr/bin/env bash
#
# Set up /etc/hosts so we can resolve all the nodes
set -e
IP_NW=$1
BUILD_MODE=$2
NUM_WORKER_NODES=$3
MASTER_IP_START=$4
NODE_IP_START=$5
if [ "$BUILD_MODE" = "BRIDGE" ]
then
# Determine machine IP from route table -
# Interface that routes to default GW that isn't on the NAT network.
MY_IP=""
for i in $(seq 1 10);
do
echo "[#1] Trying to set MY_IP"
sleep 1
MY_IP="$(ip route | grep default | grep -Pv '10\.\d+\.\d+\.\d+' | awk '{ print $9 }')"
if [ "$MY_IP" != "" ]; then
break
fi
done
if [ "$MY_IP" == "" ]; then
echo "Setting MY_IP failed"
exit 1
fi
# From this, determine the network (which for average broadband we assume is a /24)
MY_NETWORK=$(echo $MY_IP | awk 'BEGIN {FS="."} ; { printf("%s.%s.%s", $1, $2, $3) }')
# Create a script that will return this machine's IP to the bridge post-provisioner.
cat < /usr/local/bin/public-ip
#!/usr/bin/env sh
echo -n $MY_IP
EOF
chmod +x /usr/local/bin/public-ip
else
# Determine machine IP from route table -
# Interface that is connected to the NAT network.
MY_IP="$(ip route | grep "^$IP_NW" | awk '{print $NF}')"
MY_NETWORK=$IP_NW
fi
# Remove unwanted entries
sed -e '/^.*ubuntu-jammy.*/d' -i /etc/hosts
sed -e "/^.*${HOSTNAME}.*/d" -i /etc/hosts
# Export PRIMARY IP as an environment variable
echo "PRIMARY_IP=${MY_IP}" >> /etc/environment
[ "$BUILD_MODE" = "BRIDGE" ] && exit 0
# Update /etc/hosts about other hosts (NAT mode)
echo "${MY_NETWORK}.${MASTER_IP_START} controlplane" >> /etc//hosts
for i in $(seq 1 $NUM_WORKER_NODES)
do
num=$(( $NODE_IP_START + $i ))
echo "${MY_NETWORK}.${num} node0${i}" >> /etc//hosts
done
================================================
FILE: kubeadm-clusters/virtualbox/ubuntu/vimrc
================================================
set nu
set ts=2
set sw=2
set et
set ai
set pastetoggle=
================================================
FILE: managed-clusters/README.md
================================================
# Managed Clusters
Knowledge of managed clusters and how to build them is *not* part of the CKA course curriculum and therefore you will not be tested on it. These labs are provided just for fun and because people ask how to create them in KodeKloud playgrounds.
Managed clusters are where part of the cluster is managed by a major cloud provider. The control plane is *always* managed in this type of cluster, meaning that you do not have to configure control plane nodes, high availability or manage etcd - all this is "managed" for you for a fixed cost per hour the cluster is in existence.
Depending on the options chosen when creating, the worker nodes may be unmanaged, part or fully managed. However in the KodeKloud playgrounds only unmanaged nodes are available so that is what we will be creating.
We have some instructions for building managed clusters in KodeKloud playgrounds below. These builds fit within the IAM constraints of our playgrounds.
**NOTE** If you are deploying a cluster as part of a lab exercise in one of the other courses, then wherever this guide tells you to run commands in a CloudShell terminal you must instead run those commands in the lab's terminal.
* AWS EKS
These builds avoid the use of `eksctl`, managed node groups and EKS Auto Mode, none of which are supported by playground.
* [Manual build using the console](./eks/console/README.md)
* [Automated build using Terraform](https://github.com/kodekloudhub/amazon-elastic-kubernetes-service-course/blob/main/docs/playground.md)
* Azure AKS
* [Manual build using the console](./aks/console/README.md)
* [Automated build using Terraform](./aks/terraform/README.md) - From Azure CloudShell. Nothing to install on your laptop
* [Automated build using Terraform](./aks/terraform_local/README.md) - From your laptop. Requires you to install terraform, git, kubectl, azure client.
* Google GKE
* [Manual build using the console](./gke/console/README.md)
* [Automated build using Terraform](./gke/terraform/README.md)
================================================
FILE: managed-clusters/aks/console/README.md
================================================
# AKS cluster using the Console
Create an AKS cluster using the KodeKloud Azure Playground's console. [Click here](./docs/01-sign-in.md) to get started!
================================================
FILE: managed-clusters/aks/console/docs/01-sign-in.md
================================================
# Provision lab
Last updated: March 2024
### Provision your Azure playground lab
Prerequisite: Pro (or higher) KodeKloud subscription
Access this link: https://kodekloud.com/playgrounds/playground-azure and click `START LAB` to request a new Azure Playground instance. After a few seconds, you will receive your temporary Azure portal account like this:
Copy the **Console Link** by pressing the copy button to the right of it, paste to your browser and sign in with the given credentials. If you are given a choice of accounts like in the following dialog, select **Use another account**
1. At the sign-in box, copy the `Username` and paste it there, then press Next.
1. Now copy the `Password` and paste it as Temporary Access Pass.
Skip all the usage and Start a Tour stuff, unless you're interested in that.
Next: [Creating the Service](./02-create-service.md)
================================================
FILE: managed-clusters/aks/console/docs/02-create-service.md
================================================
# Create the Service
From the Azure portal dashboard, select **Kubernetes Services** or search for it in the search bar.
Next: [Create Cluster](./03-create-cluster.md)
================================================
FILE: managed-clusters/aks/console/docs/03-create-cluster.md
================================================
# Create Cluster
Then select “Create a Kubernetes cluster”
Next: [Configure Cluster](./04-creation-form.md)
================================================
FILE: managed-clusters/aks/console/docs/04-creation-form.md
================================================
# Configure Cluster
Now fill in the configuration form
### Project Details
| Field | Value |
|-------|-------|
| Subscription | *Leave unchanged* |
| Resource Group | From the drop-list choose the only available option, which starts with `ODL-azure` |
### Cluster Details
Set the following fields. Leave all others unchanged
| Field | Value |
|-------|-------|
| Cluster preset configuration | `Dev/Test` |
| Kubernetes Cluster Name | `kodekloud-demo` |
| Region | `(US) East US` |
Next: [Node Pools](./05-node-pools.md)
================================================
FILE: managed-clusters/aks/console/docs/05-node-pools.md
================================================
# Node Pools
1. Select **Node Pools** from the tab strip at the top.
Note that this menu may also appear on the left:
1. To comply with playground restrictions, we need to change some configuration. Click on the value in the **Node size** column...
1. You are now in the **Update Node Pool** screen
1. Click on **Choose a size**
And from the selection screen, choose `D2s_v3`
Press the `Select` button that appears at the bottom of the screen to return to **Update Node Pool**.
1. Next to **Scale Method** click in `Manual` radio button, then change **Node count** to `2`
1. Press the `Update` button at the bottom of the screen to complete pool configuration.
Next: [Networking](./06-networking.md)
================================================
FILE: managed-clusters/aks/console/docs/06-networking.md
================================================
# Networking
Now move on to the networking section
Here we are required to set a DNS prefix. DNS prefixes should be globally unique, and you never know if somebody else is following this guide at the same time as you! Thus we cannot use a fixed name here.
We can create a unique enough name by using `kodekloud-demo-` followed by the digits of the login ID you were given by the lab.
Leave all other settings as default.
Next: [Monitoring](./07-monitoring.md)
================================================
FILE: managed-clusters/aks/console/docs/07-monitoring.md
================================================
# Monitoring
Proceed directly to **Monitoring**. There is nothing to be set for **Integrations**
De-select *all* checkboxes to turn off monitoring.
Next: [Review and Create](./08-review-and-create.md)
================================================
FILE: managed-clusters/aks/console/docs/08-review-and-create.md
================================================
# Review and Create
Press the `Review and Create` button at the bottom of the browser window. It should think about it for a few seconds then produce a summary screen with a `Create` button at the bottom.
Press the `Create` button. It will take several minutes to provision, so go and make tea/coffee :smile:!
When it completes, you should see this
Click on `Go to resource`
You may see an error similar to the following, but it is an expected issue in the Azure Playground and doesn’t impact any AKS features, so just ignore it.
> The client 'odl_user_1278229@cloudlabs4kodeKloud.onmicrosoft.com' with object id 'da188846-792f-40be-8c62-ed65018e3a6e' does not have authorization to perform action 'Microsoft.Resources/subscriptions/resourceGroups/read' over scope '/subscriptions/1ca63aff-186a-4e2b-b3bc-f7dddf1d8969/resourceGroups/MC_ODL-azure-1278229_kodekloud-demo_eastus' or the scope is invalid. If access was recently granted, please refresh your credentials.
Next: [Connect to Cluster](./09-connect.md)
================================================
FILE: managed-clusters/aks/console/docs/09-connect.md
================================================
# Connect to Cluster
We will connect to the cluster using the Azure CloudShell.
When you click on **Connect**, it will pop up a window to the right. Click on the **Open Cloud Shell** link
1. At the popup that opens, select `Bash`
1. At the next popup, select the following
* Mount storage account
* Subscription - select from the drop-down. There should be only one choice, something like `azurekmlprod`
* Press `Apply`
1. At the `Mount Storage Account` popup, select `I want to create a storage account` and press `Next`
1. Now select the following from the `Create storage account` popup
* `Subscription` - Do not change this.
* `Resource gorup` - select from the drop-down. There should be only one choice.
* `Storage account name` - Anything you like or random characters typed from your keyboard. At least 16.
* `File share` - Anything you like or random characters typed from your keyboard. At least 16.
* `Region` - Select `(US) East US` from the dropdown
* Press `Create`, wait for it to deploy, then for the bash prompt to appear.
The cloud shell window will open at the bottom of your browser window and will automatically run the two commands that are required to set up cluster connectivity.
Now you can run commands against the cluster, and you're done!
```text
odl_user [ ~ ]$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
aks-agentpool-33115789-vmss000000 Ready agent 27m v1.28.5
aks-agentpool-33115789-vmss000001 Ready agent 26m v1.28.5
odl_user [ ~ ]$ kubectl run nginx --image=nginx
pod/nginx created
odl_user [ ~ ]$ kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 8s
odl_user [ ~ ]$
```
================================================
FILE: managed-clusters/aks/terraform/README.md
================================================
# AKS with Terraform
Last updated: July 2025
In this guide, we will deploy an AKS cluster which is compliant with the KodeKloud playground security and usage policies using terraform from within Cloud Shell. Running Cloud Shell from within the Azure console has the advantage that all the authentication and project details are already set in the environment.
[Click here](./docs/01-sign-in.md) to get started!
================================================
FILE: managed-clusters/aks/terraform/docs/01-sign-in.md
================================================
# Provision lab
Last updated: March 2024
### Provision your Azure playground lab
Prerequisite: Pro (or higher) KodeKloud subscription
Access this link: https://kodekloud.com/playgrounds/playground-azure and click `START LAB` to request a new Azure Playground instance. After a few seconds, you will receive your temporary Azure portal account like this:
Copy the **Console Link** by pressing the copy button to the right of it, paste to your browser and sign in with the given credentials. If you are given a choice of accounts like in the following dialog, select **Use another account**
1. At the sign-in box, copy the `Username` and paste it there, then press Next.
1. Now copy the `Password` and paste it as Temporary Access Pass.
Skip all the usage and Start a Tour stuff, unless you're interested in that.
Next: [Open a Cloud Shell](./02-cloudshell.md)
================================================
FILE: managed-clusters/aks/terraform/docs/02-cloudshell.md
================================================
# Open Cloudshell
1. Press the CloudShell button at the top right
1. At the popup that opens, select `Bash`
1. At the next popup, select the following
* Mount storage account
* Subscription - select from the drop-down. There should be only one choice, something like `azurekmlprod`
* Press `Apply`
1. At the `Mount Storage Account` popup, select `I want to create a storage account` and press `Next`
1. Now select the following from the `Create storage account` popup
* `Subscription` - Do not change this.
* `Resource gorup` - select from the drop-down. There should be only one choice.
* `Storage account name` - Anything you like or random characters typed from your keyboard. At least 16.
* `File share` - Anything you like or random characters typed from your keyboard. At least 16.
* `Region` - Select `(US) East US` from the dropdown
* Press `Create`, wait for it to deploy, then for the bash prompt to appear.
Next: [Install Terraform](./03-install-terraform.md)
================================================
FILE: managed-clusters/aks/terraform/docs/03-install-terraform.md
================================================
# Install Terraform
Here we will install terraform and pull the configuration we are going to use down from Github.
Perform the following commands in the Cloudshell terminal
## Install terraform
```bash
terraform_version=$(curl -s https://checkpoint-api.hashicorp.com/v1/check/terraform | jq -r -M '.current_version')
curl -O "https://releases.hashicorp.com/terraform/${terraform_version}/terraform_${terraform_version}_linux_amd64.zip"
unzip terraform_${terraform_version}_linux_amd64.zip
mkdir -p ~/bin
mv terraform ~/bin/
terraform version
```
# Clone the repository
```bash
git clone https://github.com/kodekloudhub/certified-kubernetes-administrator-course.git
cd certified-kubernetes-administrator-course/managed-clusters/aks/terraform
```
Next: [Deploy Cluster](./04-deploy-cluster.md)
================================================
FILE: managed-clusters/aks/terraform/docs/04-deploy-cluster.md
================================================
# Deploy the cluster
1. Now we can deploy the cluster
```bash
terraform init
terraform plan
terraform apply
```
This will take around 5 minutes to complete. In the console, if you navigate to `Kubernetes Service` you can see it creating.
1. Connect to the cluster.
When the terraform completes, you will see someting like the following. Copy the two `az` commands it is displaying and paste them to the terminal. Note that the subscription ID and resource group name will be different to what is displayed on this page.
```
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
Outputs:
commands = < control‑plane tier Free
sku_tier = "Free"
# RBAC with local accounts (the portal default)
role_based_access_control_enabled = true
local_account_disabled = false
# System‑assigned managed identity for the cluster
identity {
type = "SystemAssigned"
}
dns_prefix = "${var.cluster_name}-${random_string.dns.result}"
############################################
# Default system node‑pool (manual scaling) #
############################################
default_node_pool {
name = "sysnp" # 3–12 chars, starts with letter
vm_size = "Standard_D2s_v3"
node_count = 2 # manual scaling, fixed at 2
os_sku = "Ubuntu"
upgrade_settings {
drain_timeout_in_minutes = "0"
max_surge = "10%"
node_soak_duration_in_minutes = "0"
}
}
}
# Output the AZ commands user needs to run in order to access cluster.
output "commands" {
value = join("\n", [
"",
"Now run the following commands to gain kubectl access to the cluster",
"",
"az account set --subscription ${data.external.environment.result["subscription_id"]}",
"az aks get-credentials --resource-group ${data.external.environment.result["resource_group_name"]} --name ${var.cluster_name} --overwrite-existing",
""
])
}
================================================
FILE: managed-clusters/aks/terraform_local/README.md
================================================
# AKS with Terraform from local machine
Last updated: July 2025
In this guide, we will deploy an AKS cluster which is compliant with the KodeKloud playground security and usage policies using terraform from local users machine. So we will be accessing KodeKloud Playgrounds Using Terraform from Local Machine and deploying an AKS. Only thing you have to take care is to refer location and resource group from locals
[Click here](./docs/01-sign-in.md) to get started!
================================================
FILE: managed-clusters/aks/terraform_local/docs/01-sign-in.md
================================================
# Provision lab
Last updated: March 2024
### Provision your Azure playground lab
Prerequisite: Pro (or higher) KodeKloud subscription
Access this link: https://kodekloud.com/playgrounds/playground-azure and click `START LAB` to request a new Azure Playground instance. After a few seconds, you will receive your temporary Azure portal account like this:
Copy the **Console Link** by pressing the copy button to the right of it, paste to your browser and sign in with the given credentials. If you are given a choice of accounts like in the following dialog, select **Use another account**
1. At the sign-in box, copy the `Username` and paste it there, then press Next.
1. Now copy the `Password` and paste it as Temporary Access Pass.
1. Copy the `Application Client ID`, this value need to be set for variable `TF_VAR_ARM_CLIENT_ID`
1. Copy the `Client Secret`, this value need to be set for variable `TF_VAR_ARM_CLIENT_SECRET`
Skip all the usage and Start a Tour stuff, unless you're interested in that.
1. Copy `Subscription ID` → Found in Azure Portal → Subscriptions → Subscription ID, this value need to be set for variable `TF_VAR_ARM_SUBSCRIPTION_ID`
2. Copy `Tenant ID` → Found in Azure Portal → Microsoft Entra ID → Overview → Tenant ID, this value need to be set for variable `TF_VAR_ARM_TENANT_ID`
Next: [Install softwares](./02-install-softwares.md)
================================================
FILE: managed-clusters/aks/terraform_local/docs/02-install-softwares.md
================================================
# Install Software
For best results install required software using package managers.
* Windows - Use [chocolatey package manager](https://github.com/kodekloudhub/community-faq/blob/main/docs/how-tos/howto-package-management-on-windows.md)
* macOS - Use [HomeBrew](https://brew.sh/)
* Linux - Use the builtin package managers e.g. `apt`, `yum` etc.
Install the following if you don't aloready have it.
* [terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli#install-terraform)
* `kubectl`
- [macOS](https://kubernetes.io/docs/tasks/tools/install-kubectl-macOS/)
- [Linux](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/)
- [Windows](https://kubernetes.io/docs/tasks/tools/install-kubectl-windows/)
* `azure-cli`
* [macOS](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-macos?view=azure-cli-latest)
* [Linux](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?view=azure-cli-latest)
* Windows
```
choco install -y azure-cli
```
# Verify the installations
Open a new shell and verify that each of the above is correctly installed:
1. To verify Terraform, open powershell and type
```
terraform version
```
1. To verify kubectl version, open powershell and type
```
kubectl version --client
```
1. To verify Azure CLI version, open powershell and type
```
az version
```
Note that you may get an error here on Windows. If you do, then fix it by running the following in a command prompt started `as Administrator`
```
"C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe" -m pip install pywin32 --force-reinstall
```
# Clone the repository
```bash
git clone https://github.com/kodekloudhub/certified-kubernetes-administrator-course.git
cd certified-kubernetes-administrator-course/managed-clusters/aks/terraform_local
```
Once you have verified the software packages and cloned the repository, please proceed with next step.
Next: [Setting Variable](./03-setting-variable.md)
================================================
FILE: managed-clusters/aks/terraform_local/docs/03-setting-variable.md
================================================
# Setting Up Environment Variables for Terraform
Now we need to set various envornment variables requirede by terraform which concern authentication with Azure. These are the values gathered earlier in this guide.
---
## Step 1: Navigate to the Project Directory
Open a terminal (bash for Linux or macOS, or PowerShell/GitBash for Windows) in the folder where this repository has been downloaded or cloned.
---
## Step 2: Set Environment Variables
These variables are required for authenticating with Azure.
### **Using Linux, MacOS or Windows Git Bash**
Run the following commands (replace values with your actual credentials):
```bash
export TF_VAR_ARM_CLIENT_ID=
export TF_VAR_ARM_CLIENT_SECRET=
export TF_VAR_ARM_TENANT_ID=
export TF_VAR_ARM_SUBSCRIPTION_ID=
```
### Using **Powershell (Windows)**
Run the following commands (quotes required):
```pwsh
$env:TF_VAR_ARM_CLIENT_ID = ""
$env:TF_VAR_ARM_CLIENT_SECRET = ""
$env:TF_VAR_ARM_TENANT_ID = ""
$env:TF_VAR_ARM_SUBSCRIPTION_ID = ""
```
---
## Step 3: Configure main.tf for Your Shell
Terraform uses the external data source to fetch environment variables via a script. Use the block corresponding to your shell.
* If Using PowerShell (Windows)
Uncomment the following block in `main.tf` and comment out the Bash block if present.:
```
data "external" "environment" {
program = ["powershell", "-ExecutionPolicy", "Bypass", "-File", "${path.module}/environment.ps1"]
}
```
* For all other environments
Uncomment the following block in `main.tf` and comment out the PowerShell block if present.:
```
data "external" "environment" {
program = ["bash", "${path.module}/environment.sh"]
}
```
* Only one external block (Git Bash or PowerShell) should be active at a time.
* These environment variables are temporary and available only in the current terminal session.
Next: [Deploy Cluster](./04-deploy-cluster.md)
================================================
FILE: managed-clusters/aks/terraform_local/docs/04-deploy-cluster.md
================================================
# Deploy the cluster
1. Now we can deploy the cluster
```powershell or Git bash
terraform init
terraform plan
terraform apply
```
This will take around 5 minutes to complete. In the console, if you navigate to `Kubernetes Service` you can see it creating.
1. Connect to the cluster.
When the terraform completes, you will see someting like the following. Copy the two `az` commands it is displaying and paste them to the terminal. Note that the subscription ID and resource group name will be different to what is displayed on this page.
```
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
Outputs:
commands = < /dev/null 2>&1
# Get resource group starting with "kml"
RG_NAME=$(az group list --query "[?starts_with(name, 'kml')].name | [0]" -o tsv)
# Get location of that resource group
LOCATION=$(az group show --name "$RG_NAME" --query location -o tsv)
# Output in JSON format for Terraform
echo "{
\"location\": \"$LOCATION\",
\"resource_group_name\": \"$RG_NAME\"
}"
================================================
FILE: managed-clusters/aks/terraform_local/main.tf
================================================
terraform {
required_version = ">= 1.7.0"
required_providers {
azurerm = {
source = "hashicorp/azurerm"
version = "4.26.0"
}
random = {
source = "hashicorp/random"
version = "~> 3.6"
}
external = {
source = "hashicorp/external"
version = "~> 2.3"
}
}
}
variable "cluster_name" {
type = string
default = "kodekloud-demo"
}
# For KodeKloud Connections - Starting
# If running on Linux, MacOS or Git Bash on Windows, use the below block , else comment this section.
#data "external" "environment" {
# program = ["bash", "${path.module}/environment.sh"]
#}
# If running on Powershell on Windows, use the below block , else comment this section.
data "external" "environment" {
program = ["powershell", "-ExecutionPolicy", "Bypass", "-File", "${path.module}/environment.ps1"]
}
locals {
subscription_id = var.ARM_SUBSCRIPTION_ID
location = data.external.environment.result["location"]
resource_group_name = data.external.environment.result["resource_group_name"]
}
output "subscription_id" {
value = local.subscription_id
}
output "resource_group_name" {
value = local.resource_group_name
}
output "location" {
value = local.location
}
# For KodeKloud Connections - Ending
variable "ARM_CLIENT_ID" {
type = string
description = "Azure Client ID"
}
variable "ARM_CLIENT_SECRET" {
type = string
description = "Azure Client Secret"
}
variable "ARM_TENANT_ID" {
type = string
description = "Azure Tenant ID"
}
variable "ARM_SUBSCRIPTION_ID" {
type = string
description = "Azure Subscription ID"
}
# Configure Azure provider
provider "azurerm" {
resource_provider_registrations = "none"
features {}
client_id = var.ARM_CLIENT_ID
client_secret = var.ARM_CLIENT_SECRET
tenant_id = var.ARM_TENANT_ID
subscription_id = var.ARM_SUBSCRIPTION_ID
}
# Generate a random suffix for cluster's DNS prefix
resource "random_string" "dns" {
length = 6
upper = false
lower = false
numeric = true
special = false
}
# Deploy the cluster
resource "azurerm_kubernetes_cluster" "aks" {
name = var.cluster_name
location = local.location
resource_group_name = local.resource_group_name
# Preset: Dev/Test –> control‑plane tier Free
sku_tier = "Free"
# RBAC with local accounts (the portal default)
role_based_access_control_enabled = true
local_account_disabled = false
# System‑assigned managed identity for the cluster
identity {
type = "SystemAssigned"
}
dns_prefix = "${var.cluster_name}-${random_string.dns.result}"
############################################
# Default system node‑pool (manual scaling) #
############################################
default_node_pool {
name = "sysnp" # 3–12 chars, starts with letter
vm_size = "Standard_D2s_v3"
node_count = 2 # manual scaling, fixed at 2
os_sku = "Ubuntu"
upgrade_settings {
drain_timeout_in_minutes = "0"
max_surge = "10%"
node_soak_duration_in_minutes = "0"
}
}
}
# Output the AZ commands user needs to run in order to access cluster.
output "commands" {
value = join("\n", [
"",
"Now run the following commands to gain kubectl access to the cluster",
"",
"az account set --subscription ${local.subscription_id}",
"az aks get-credentials --resource-group ${local.resource_group_name} --name ${var.cluster_name} --overwrite-existing",
""
])
}
================================================
FILE: managed-clusters/eks/console/README.md
================================================
# EKS cluster using the Console
Last updated: December 2024
Create an EKS cluster using the KodeKloud AWS Playground's console. Due to policy restrictions in the playground, it is important to follow these instructions *to the letter*. Do not change any names for things like the cluster or role names or it will not work.
[Click here](./docs/01-sign-in.md) to get started!
================================================
FILE: managed-clusters/eks/console/docs/01-sign-in.md
================================================
# Provision lab
Last updated: December 2024
Be aware that of the three managed cluster types, EKS is by far the most complicated to build as in the number of steps required to complete it. Please pay special attention to each, as if you make a mistake then you will see errors. *This does work* and when it doesn't, it will be because you missed something and you will probably be best ending the lab and starting over! Having said that, AWS do from time to time reorganize the console screens and workflow. If you find that what you see in the console deviates from what is in this guide, then please do call it out on Discord or the [Community Forum](https://kodekloud.com/community/c/kubernetes/6).
### Provision your AWS playground lab
Prerequisite: Premium KodeKloud subscription
Access this link: https://kodekloud.com/playgrounds/playground-aws and click `LAUNCH NOW` to request a new AWS Playground instance. After a few seconds, you will receive your temporary AWS portal account like this:
Copy the **Console Link** by pressing the copy button to the right of it, paste to your browser and sign in with the given credentials. At the login screen:
* Paste the given username to the `IAM user name` field.
* Paste the password to the `Password` field.
* Uncheck `Remember this account` checkbox if it is checked.
* Press `Sign in` button.
Once you have signed in, ensure the region is set to `us-east-1` (N. Virginia). If it is not displaying this, click on the region dropdown (indicated by arrow) and select `US East (N. Virginia) us-east-1` from the menu.
Next: [Creating the Service Role](./02-create-service-role.md)
================================================
FILE: managed-clusters/eks/console/docs/02-create-service-role.md
================================================
# Create the Service Role
Note that various information pop-ups may appear at the top of the console screen as you do things and may obscure other items. Dismiss these when they appear.
Before creating an EKS cluster you need to create a Cluster service IAM (Identity and Access Management) role. This grants permissions for the EKS service to access AWS APIs on your behalf.
1. Navigate to the IAM console
1. Click in the search box and type `iam`
1. Click on `IAM` in the result list
1. Click on `Roles` in the left menu
1. Click on orange `Create Role` button towards the top right
1. On the next screen `AWS Service` should be selected. If not, click on it to select.
1. In the `Use case` box, click on `Choose a service or use case` and type `eks`. EKS should come up in the list. Click on it.
1. In the list of radio buttons that will now appear, ensure `EKS - Cluster` is selected. It should now look like this:
1. Click on orange `Next` button at the bottom right. You may need to scroll down to get to it.
1. Press `Next` at the Add Permission screen. There are no additional permission required.
1. Finally name the role as `eksClusterRole`, scroll to end and press orange `Create role` button
Prev: [Sign in](./01-sign-in.md)
Next: [Configure Cluster](./03-configure-cluster.md)
================================================
FILE: managed-clusters/eks/console/docs/03-configure-cluster.md
================================================
# Configure Cluster
Now we will configure the cluster options
1. Navigate to the EKS console
1. Click in the search box and type `eks`
1. Click on `Elastic Kubernetes Service` in the result list
1. Click on orange `Create cluster` button. This will open a list, and from that, choose `Create
1. In the Configure Cluster page:
1. Select `Custom Configuration`
1. Ensure `EKS Auto Mode` is switched OFF
1. Enter the name as `demo-eks` and select the cluster service role created above.
1. Enable ConfigMap for authentication (it's needed to join nodes later)
1. Then scroll to the end and press `Next` button.
Prev: [Create Service Role](./02-create-service-role.md)
Next: [Networking](./04-networking.md)
================================================
FILE: managed-clusters/eks/console/docs/04-networking.md
================================================
# Networking
1. Select VPC - there should only be one choice, the Default VPC.
1. Select values for the Subnets fields. There needs to be at least two subnets selected, and we don't need more than 3. Ensure that `us-east-1e` is *not* one of the selected subnets. If there are less than 2 subnets available, you will either need to go to VPC console and add some (beyond scope of this guide), or restart the playground until you get 2 or more subnets.
1. Scroll to end and press `Next`.
1. At the following page (Observability), press `Next` again as there is nothing to do here.
1. At the following page (Select add-ons), press `Next` again as there is nothing to do here either.
1. At the following page (Configure selected add-ons settings), press `Next` again as there is nothing to do here either.
Prev: [Configure Cluster](./03-configure-cluster.md)
Next: [Create Cluster](./05-create-cluster.md)
================================================
FILE: managed-clusters/eks/console/docs/05-create-cluster.md
================================================
# Create Cluster
Now you should be on the Review and Create page. Here we will launch the control plane. No nodes are ready yet - this comes later.
1. Review the information that you entered or selected on the previous pages. If you need to make changes, choose an `Edit` button next to the section you want to change. After that, scroll to the end and press `Create`. The Status field shows "Creating" while the cluster is provisioned. This will take *at least* 10 minutes, so go and make a cup of tea! You may need to press the refresh button periodically (indicated by arrow) to ensure the information is up to date.
1. Wait for the control plane to deploy. When the control plane is deployed, the status will change from "Creating" to "Active"
1. **IMPORTANT**: if you are building the cluster as part of a lab or KKE task, you must enter all the following commands at the *lab* terminal, not CloudShell, or the grader will not be able to connect to your cluster.
Once the cluster is created, enable kubectl to communicate with your cluster by adding a new context to the kubectl config file by executing the following command in CloudShell, which you can invoke by pressing the cloudshell button indicated by the number `3` in the image above. This will open a cloudshell prompt below the EKS console into which you can paste and run the following:
```
aws eks update-kubeconfig --region us-east-1 --name demo-eks
```
Check it is connecting. You should see some resources. All pods are in Pending state because we have yet to create nodes to run them on.
```
kubectl get all -A
```
Prev: [Networking](./04-networking.md)
Next: [Add nodes](./06-nodes.md)
================================================
FILE: managed-clusters/eks/console/docs/06-nodes.md
================================================
# Add Cluster Nodes
Now we will add some *unmanaged* nodes to the cluster. AWS Playground does not support managed node groups.
1. Create an SSH keypair for the nodes to use
1. Navigate to the EC2 console by typing `ec2` in the search box at the top, and then selecting it from the list.
1. Scroll down the menu on the left until you see `Key Pairs`, then click that to enter the key pairs console.
1. Click the orange `Create key pair` button at the top right
1. Fill out the form using `node-key-pair` as name, `RSA` as key pair type and `.pem` as private key format
1. Press the `Create key pair` button
1. Navigate to the CloudFormation console, again using the search box. CloudFormation is AWS's native Infrastructure as Code, and we will use a pre-prepared template to define worker nodes.
1. Press the orange `Create stack` button
1. Select Choose an existing template from Amazon S3 URL and use the following URL: `https://s3.us-west-2.amazonaws.com/amazon-eks/cloudformation/2022-12-23/amazon-eks-nodegroup.yaml`
1. Scroll to end of page and press `Next`
1. Now we fill in the parameters required to deploy the nodes
1. `Stack Name`: `eks-cluster-stack`
1. `ClusterName`: `demo-eks`
1. `ClusterControlPlaneSecurityGroup`: Click in the box and select the one with a name that contains `eks-cluster-sg`
1. `NodeGroupName`: `eks-demo-node`
1. `KeyName`: (you will likely need to scroll down to find this) - `node-key-pair` as created above.
1. `VpcId`: Click in the box and select the only entry that is there
1. `Subnets`. Sadly the drop-list does not show the AZ of the nets so it is fiddly to determine which ones to select
1. Open the [subnets console](https://us-east-1.console.aws.amazon.com/vpcconsole/home?region=us-east-1#subnets:) in another browser tab or window.
1. Click on each subnet to view its information in the lower pane.
1. Note the IPv4 CIDR for the ones that are in the subnets you selected when configuring cluster networking.
1. Choose these subnets back in the cloudformation console
1. Leave other settings as defaults
It should now look something like this
1. Press `Next`
1. Nothing to do on the following screen. Scroll to end and press `Next`
1. Now you are on the final screen. Scroll to the end and check the acknowledge box, then press `Submit`
1. Wait for stack creation to complete which will take a few minutes. You may need to press the Refresh button a few times until it gets to `CREATE COMPLETE`
1. Now select the `Outputs` tab and note down the value of `NodeInstanceRole`. You need this when you join your Amazon EKS nodes.
Prev: [Create Cluster](./05-create-cluster.md)
Next: [Join Worker Nodes](./07-join-nodes.md)
================================================
FILE: managed-clusters/eks/console/docs/07-join-nodes.md
================================================
# Join Worker Nodes
Now that CloudFormation has created the EC2 instances for the worker nodes, we need to join them to the cluster. Once again, if this is a lab or KKE task, the following must be done at the *lab* terminal.
1. Return to the CloudShell terminal. If you closed it, press the button at the top to reopen it. If it tells you the session is closed due to inactivity, hit Enter to restart it.
1. Download the node configmap
```
curl -O https://s3.us-west-2.amazonaws.com/amazon-eks/cloudformation/2020-10-29/aws-auth-cm.yaml
```
1. Now edit it in `vi`
```
vi aws-auth-cm.yaml
```
Where it says `rolearn: `, you need to delete `` and paste in the value for `NodeInstanceRole` you got from the previous section. When you are done, it should look something like this (though the actual value will be different for you).
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: aws-auth
namespace: kube-system
data:
mapRoles: |
- rolearn: arn:aws:iam::851725221429:role/eks-cluster-stack-NodeInstanceRole-8OYkncRSa4gA
username: system:node:{{EC2PrivateDNSName}}
groups:
- system:bootstrappers
- system:nodes
```
1. Save and exit from vi, then apply the configmap
```
kubectl apply -f aws-auth-cm.yaml
```
It will take a minute or so for all nodes to join the cluster
```
kubectl get nodes
```
Congratulations! Your cluster is now up and you can create resources.
Prev: [Add Cluster Nodes](./06-nodes.md)
Next: [Accessing Node Port services](./08-node-port.md)
================================================
FILE: managed-clusters/eks/console/docs/08-node-port.md
================================================
# Accessing Node Port services
In order for you to be able to see any NodePort services you create, we must edit the security group applied to the nodes in order to permit your laptop to access them. In AWS, a security group acts like a firewall. The default configuration for this created in the prvious step only permits access between the control plane and the nodes. We will now edit this to allow your broadband public IP to also access the nodes on the port range for NodePort services.
## Edit the node security group
1. Navigate to the EC2 console using the search box.
1. In the menu on the left, scroll down till you find `Security groups` and click on it.
1. In the list of security groups, click on the `eks-cluster-stack-NodeSecurityGroup`.
1. In the lower pane that appears, click on `Edit inbound rules`.
1. On the edit rules page that now comes up, click on `Add Rule` button which is at the end of the `Inbound Rules` list.
1. Fill in the new security group rule
* `Type`: `Custom TCP`
* `Port range`: `30000 - 32768` which is the default range for node ports as you should know from the course.
* `Source`: `My IP` It will automatically determine your broadband public IP. You can find this yourself by browsing http://checkip.amazonaws.com
1. Press the orange `Save rules` button.
## Create a test service
1. While you are still in the EC2 console, navigate to `Instances` in the menu on the left and click on it
1. Note the `Public IP` addresses of the nodes. You will need to use any one of these to connect to your nodeport service.
1. Return to cloudshell and create a service
1. Create pod and service
```
kubectl run nginx --image nginx --expose --port=80
```
1. Edit the service and change it to nodeport.
```
kubectl edit service nginx
```
Make the manifest look like this (I have only shown the relevant sections of the service manifest here)
```yaml
ports:
- port: 80
protocol: TCP
targetPort: 80
nodePort: 30080 #<- Add this
selector:
run: nginx
sessionAffinity: None
type: NodePort #<- Edit this from ClusterIP to NodePort
```
1. Now you can view your service in your browser by building the URL from the public IP address you got from the EC2 console, and the node port `30080`. In this example it is `http://44.198.158.250:30080`, but for you the IP address will be different. Paste the URL you have formed into your browser. You should see the nginx welcome page.
Prev: [Join Nodes](./07-join-nodes.md)
================================================
FILE: managed-clusters/gke/console/README.md
================================================
# GKE cluster using the Console
Create a GKE cluster using the KodeKloud GCP Playground's console. [Click here](./docs/01-sign-in.md) to get started!
================================================
FILE: managed-clusters/gke/console/docs/01-sign-in.md
================================================
# Provision lab
Last updated: March 2025
In this guide, we will deploy a GKE cluster which is compliant with the KodeKloud playground security and usage policies. Note that you must follow this guide *precisely* or you may get permission errors.
Since this is a Google product, it will work best in Chrome browser. Other mainstream browsers *should* be fine.
### IMPORTANT
Note that if you are already signed into your browser with a Google account, it may try to use that and you will get an error and/or sign yourself out of your Google account. If you already have a personal GCP account and you are signed into Google, then it *will use this automatically* and not connect you to the playground! If either of these apply to you, then use an incognito window to sign into the playground account.
### Provision your GCP playground lab
Prerequisite: Premium KodeKloud subscription
Access this link: https://kodekloud.com/playgrounds/playground-google-cloud and click `START LAB` to request a new GCP Playground instance. After a few seconds, you will receive your temporary GCP portal account like this:
Copy the **Console Link** by pressing the copy button to the right of it, paste to your browser and sign in with the given credentials. Provide the username given for the lab as `email or phone` in the sign-in dialog.
In the dialog box that follows, select your country and agree to terms of service. Leave **Email updates** unchecked.
Next: [Provisioning the Cluster](./02-create-cluster.md)
================================================
FILE: managed-clusters/gke/console/docs/02-create-cluster.md
================================================
# Provisioning the Cluster
There is no longer any need to select project, or set up routes or firewall rules as directed by the [original guide published on the Community Forum](https://kodekloud.com/community/t/playground-series-how-to-create-a-managed-kubernetes-cluster-with-google-kubernetes-engine/230314). These are now all in place when you start, therefore you can get straight onto configuring the cluster.
Note that as you navigate around the GCP site, various blue information pop-ups may appear. Dismiss these if they do.
If using a small monitor, some top menu items may be hidden behind a vertical ellipsis - `⋮`. Check for this if some options shown in the screenshots can't be found.
1. In the menu on the left, select `Kubernetes Engine`. You don't need to select any item from the submenu that pops up.
1. On the following screen, press the `CREATE CLUSTER` button. You will arrive at the Cluster Basics screen.
1. Near the top right of this screen, click on `SWITCH TO STANDARD CLUSTER`, and confirm this is what you want to do in the dialog that pops up
1. Edit the cluster name as shown below. Other names *should* be ok, but if you have issues, use this one. Ensure that you select `Regional`, and set the region to `us-west1`. Leave all other fields as defaults
We've started configuring the cluster, but still have additional things we need to set, so don't click the CREATE button at the bottom of this screen just yet. Before that, we'll configure the node pool on the next page...
Prev: [Sign in](./01-sign-in.md)
Next: [Node Setup](./03-node-setup.md)
================================================
FILE: managed-clusters/gke/console/docs/03-node-setup.md
================================================
# Node Setup
1. Click on `NODE POOLS/Default Pool` to the left. Set `Number of nodes (per zone)` to `1`. Leave all other fields as defaults.
1. Click on Nodes. Ensure `Machine Configuration` is set to `E2`. Set `Boot disk type` to `Standard persistent disk` and `Boot disk size` to `20`
The more advanced features of GKE aren't enabled in KodeKloud playgrounds, so there is no more configuration needed. Thus you are now ready to go! Press the `CREATE` button at the bottom of this screen. It will take at least 5 minutes to deploy your new cluster. It will take you to this screen. You may have to scroll it down to see the cluster at the bottom. You can click on the cluster name to see a progress screen.
Prev: [Provisioning Cluster](./02-create-cluster.md)
Next: [Connect to Cluster](./04-connect.md)
================================================
FILE: managed-clusters/gke/console/docs/04-connect.md
================================================
# Connect to Cluster
1. When the cluster has deployed, then there will be a green tick in the status column. You can now click on the cluster name, unless you did already.
1. At the top of the following screen, click on `CONNECT`
1. Click on `RUN IN CLOUDSHELL`. The CloudShell pane will open at the bottom of the screen. Press the `CONTINUE` button.
1. The terminal will open and some commands will appear in it, Hit `ENTER` to execute them. Press `AUTHORIZE` on the dialog that pops up.
1. Test with `kubectl`
```text
kubectl get pods -A
```
================================================
FILE: managed-clusters/gke/terraform/README.md
================================================
# GKE with Terraform
Last updated: March 2025
In this guide, we will deploy a GKE cluster which is compliant with the KodeKloud playground security and usage policies using terraform from within Cloud Shell. Running Cloud Shell from within the GCP console has the advantage that all the authentication and project details are already set in the environment.
Since this is a Google product, it will work best in Chrome browser. Other mainstream browsers *should* be fine.
[Click here](./docs/01-sign-in.md) to get started!
================================================
FILE: managed-clusters/gke/terraform/docs/01-sign-in.md
================================================
# Provision lab
### IMPORTANT
Note that if you are already signed into your browser with a Google account, it may try to use that and you will get an error and/or sign yourself out of your Google account. If you already have a personal GCP account and you are signed into Google, then it *will use this automatically* and not connect you to the playground! If either of these apply to you, then use an incognito window to sign into the playground account.
### Provision your GCP playground lab
Prerequisite: Premium KodeKloud subscription
Access this link: https://kodekloud.com/playgrounds/playground-google-cloud and click `START LAB` to request a new GCP Playground instance. After a few seconds, you will receive your temporary GCP portal account like this:
Copy the **Console Link** by pressing the copy button to the right of it, paste to your browser and sign in with the given credentials. Provide the username given for the lab as `email or phone` in the sign-in dialog.
In the dialog box that follows, select your country and agree to terms of service. Leave **Email updates** unchecked.
Next: [Installing Terraform](./02-install-terraform.md)
================================================
FILE: managed-clusters/gke/terraform/docs/02-install-terraform.md
================================================
# Install Terraform
Here we will install terraform and pull the configuration we are going to use down from Github.
## Open Cloudshell
First, open the Cloudshell by clicking on the icon at the top right of the dashboard. A Linux terminal will open at the bottom of the screen.
Perform the following commands in the Cloudshell terminal
## Install terraform
```bash
terraform_version=$(curl -s https://checkpoint-api.hashicorp.com/v1/check/terraform | jq -r -M '.current_version')
curl -O "https://releases.hashicorp.com/terraform/${terraform_version}/terraform_${terraform_version}_linux_amd64.zip"
unzip terraform_${terraform_version}_linux_amd64.zip
mkdir -p ~/bin
mv terraform ~/bin/
terraform version
```
# Clone the repository
```bash
git clone https://github.com/kodekloudhub/certified-kubernetes-administrator-course.git
cd certified-kubernetes-administrator-course/managed-clusters/gke/terraform
```
Next: [Deploy Cluster](./03-deploy-cluster.md)
Prev: [Sign in](./01-sign-in.md)
================================================
FILE: managed-clusters/gke/terraform/docs/03-deploy-cluster.md
================================================
# Deploy the cluster
1. Export the project ID to terraform. The project ID is provided as an environment variable by Cloudshell. We need to set the appropriate terraform environment variable from it:
```bash
export TF_VAR_project_id=$GOOGLE_CLOUD_PROJECT
```
1. Now we can deploy the cluster
```bash
terraform init
terraform plan
terraform apply
```
This will take at least 5 minutes to complete. In the console, if you navigate to `Kubernetes Engine` you can see it creating.
1. Connect to the cluster.
Run the following command in the Cloudshell to authenticate kubectl to use the cluster
```bash
gcloud container clusters get-credentials kodekloud-demo-cluster --region us-west1 --project $GOOGLE_CLOUD_PROJECT
```
1. Check nodes
```bash
kubectl get nodes
```
Once all nodes are showing ready, the cluster is ready to use!
Prev: [Install terraform](./02-install-terraform.md)
================================================
FILE: managed-clusters/gke/terraform/main.tf
================================================
terraform {
required_providers {
google = {
source = "hashicorp/google"
version = ">= 4.0"
}
}
}
provider "google" {
project = var.project_id
region = "us-west1"
}
resource "google_container_cluster" "primary" {
name = "kodekloud-demo-cluster"
location = "us-west1"
initial_node_count = 1 # Number of nodes per zone
deletion_protection = false
node_config {
machine_type = "e2-medium"
disk_size_gb = 20
disk_type = "pd-standard"
oauth_scopes = [
"https://www.googleapis.com/auth/cloud-platform"
]
}
networking_mode = "VPC_NATIVE"
}
variable "project_id" {
description = "The Google Cloud project ID"
type = string
}
================================================
FILE: metrics-staging-scripts/high_cpu_node.sh
================================================
#!/bin/bash
echo cluster1 $(kubectl --context cluster1 top node --no-headers | sort -nr -k2 | head -1) > /tmp/high_cpu_node
echo cluster2 $(kubectl --context cluster2 top node --no-headers | sort -nr -k2 | head -1) >> /tmp/high_cpu_node
echo cluster3 $(kubectl --context cluster3 top node --no-headers | sort -nr -k2 | head -1) >> /tmp/high_cpu_node
echo cluster4 $(kubectl --context cluster4 top node --no-headers | sort -nr -k2 | head -1) >> /tmp/high_cpu_node
echo cluster5 $(kubectl --context cluster5 top node --no-headers | sort -nr -k2 | head -1) >> /tmp/high_cpu_node
final_value=$(cat /tmp/high_cpu_node | sort -nr -k 3 | awk '{print $1,$2}' | head -1 | tr " " ,)
if [[ $(cat /opt/high_cpu_node | grep $final_value) ]]
then
echo SUCCESS
else
echo FAIL
fi
================================================
FILE: metrics-staging-scripts/high_cpu_pod.sh
================================================
#!/bin/bash
echo cluster1 $(kubectl --context cluster1 top pods -A --no-headers | sort -nr -k3 | head -1) > /tmp/high_cpu_pod
echo cluster2 $(kubectl --context cluster2 top pods -A --no-headers | sort -nr -k3 | head -1) >> /tmp/high_cpu_pod
echo cluster3 $(kubectl --context cluster3 top pods -A --no-headers | sort -nr -k3 | head -1) >> /tmp/high_cpu_pod
echo cluster4 $(kubectl --context cluster4 top pods -A --no-headers | sort -nr -k3 | head -1) >> /tmp/high_cpu_pod
echo cluster5 $(kubectl --context cluster5 top pods -A --no-headers | sort -nr -k3 | head -1) >> /tmp/high_cpu_pod
final_value=$(cat /tmp/high_cpu_pod | sort -nr -k 4 | awk '{print $1,$2,$3}' | head -1 | tr " " ,)
if [[ $(cat /opt/high_cpu_pod | grep $final_value) ]]
then
echo SUCCESS
else
echo FAIL
fi
================================================
FILE: metrics-staging-scripts/high_cpu_pod_1.yaml
================================================
apiVersion: v1
kind: Pod
metadata:
name: frontend-stable-cka05-arch
spec:
nodeName: cluster5-controlplane
containers:
- name: fe-cka05-arch
image: vish/stress
resources:
limits:
cpu: "0.395"
requests:
cpu: "0.35"
args:
- -cpus
- "1"
================================================
FILE: metrics-staging-scripts/high_memory_node.sh
================================================
#!/bin/bash
echo cluster1 $(kubectl --context cluster1 top node --no-headers | sort -nr -k4 | head -1) > /tmp/high_memory_node
echo cluster2 $(kubectl --context cluster2 top node --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_node
echo cluster3 $(kubectl --context cluster3 top node --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_node
echo cluster4 $(kubectl --context cluster4 top node --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_node
echo cluster5 $(kubectl --context cluster5 top node --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_node
final_value=$(cat /tmp/high_memory_node | sort -nr -k 5 | awk '{print $1,$2}' | head -1 | tr " " ,)
if [[ $(cat /opt/high_memory_node | grep $final_value) ]]
then
echo SUCCESS
else
echo FAIL
fi
================================================
FILE: metrics-staging-scripts/high_memory_pod.sh
================================================
#!/bin/bash
echo cluster1 $(kubectl --context cluster1 top pods -A --no-headers | sort -nr -k4 | head -1) > /tmp/high_memory_pod
echo cluster2 $(kubectl --context cluster2 top pods -A --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_pod
echo cluster3 $(kubectl --context cluster3 top pods -A --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_pod
echo cluster4 $(kubectl --context cluster4 top pods -A --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_pod
echo cluster5 $(kubectl --context cluster5 top pods -A --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_pod
final_value=$(cat /tmp/high_memory_pod | sort -nr -k 5 | awk '{print $1,$2,$3}' | head -1 | tr " " ,)
if [[ $(cat /opt/high_memory_pod | grep $final_value) ]]
then
echo SUCCESS
else
echo FAIL
fi
================================================
FILE: metrics-staging-scripts/high_memory_pod.yaml
================================================
apiVersion: v1
kind: Pod
metadata:
name: backend-cka06-arch
namespace: default
spec:
containers:
- name: memory-demo-ctr
image: polinux/stress
resources:
requests:
memory: "200Mi"
limits:
memory: "900Mi"
command: ["stress"]
args: ["--vm", "1", "--vm-bytes", "800M", "--vm-hang", "1"]
================================================
FILE: resources/app-wl03.yaml
================================================
---
apiVersion: v1
kind: Pod
metadata:
name: app-wl03
spec:
containers:
- name: monitor-tool
image: ubuntu
args:
- "sleep"
- "5000"
resources:
requests:
memory: 1Gi
limits:
memory: 100Mi
================================================
FILE: resources/beta-logger.yaml
================================================
apiVersion: v1
kind: Pod
metadata:
name: logger
namespace: beta
spec:
containers:
- name: beta
image: busybox
args: [/bin/sh, -c, 'while true; do echo "INFO: $(date) Logger is running"; sleep 2; echo "ERROR: $(date) Logger encountered errors!"; sleep 2;done']
================================================
FILE: resources/beta-namespace.yaml
================================================
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: null
name: beta
spec: {}
status: {}
================================================
FILE: resources/calico/calico.yaml
================================================
---
# Source: calico/templates/calico-kube-controllers.yaml
# This manifest creates a Pod Disruption Budget for Controller to allow K8s Cluster Autoscaler to evict
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: calico-kube-controllers
namespace: kube-system
labels:
k8s-app: calico-kube-controllers
spec:
maxUnavailable: 1
selector:
matchLabels:
k8s-app: calico-kube-controllers
---
# Source: calico/templates/calico-kube-controllers.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: calico-kube-controllers
namespace: kube-system
---
# Source: calico/templates/calico-node.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: calico-node
namespace: kube-system
---
# Source: calico/templates/calico-config.yaml
# This ConfigMap is used to configure a self-hosted Calico installation.
kind: ConfigMap
apiVersion: v1
metadata:
name: calico-config
namespace: kube-system
data:
# Typha is disabled.
typha_service_name: "none"
# Configure the backend to use.
calico_backend: "bird"
# Configure the MTU to use for workload interfaces and tunnels.
# By default, MTU is auto-detected, and explicitly setting this field should not be required.
# You can override auto-detection by providing a non-zero value.
veth_mtu: "0"
# The CNI network configuration to install on each node. The special
# values in this config will be automatically populated.
cni_network_config: |-
{
"name": "k8s-pod-network",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "calico",
"log_level": "info",
"log_file_path": "/var/log/calico/cni/cni.log",
"datastore_type": "kubernetes",
"nodename": "__KUBERNETES_NODE_NAME__",
"mtu": __CNI_MTU__,
"ipam": {
"type": "calico-ipam"
},
"policy": {
"type": "k8s"
},
"kubernetes": {
"kubeconfig": "__KUBECONFIG_FILEPATH__"
}
},
{
"type": "portmap",
"snat": true,
"capabilities": {"portMappings": true}
},
{
"type": "bandwidth",
"capabilities": {"bandwidth": true}
}
]
}
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: bgpconfigurations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BGPConfiguration
listKind: BGPConfigurationList
plural: bgpconfigurations
singular: bgpconfiguration
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
description: BGPConfiguration contains the configuration for any BGP routing.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: BGPConfigurationSpec contains the values of the BGP configuration.
properties:
asNumber:
description: 'ASNumber is the default AS number used by a node. [Default:
64512]'
format: int32
type: integer
bindMode:
description: BindMode indicates whether to listen for BGP connections
on all addresses (None) or only on the node's canonical IP address
Node.Spec.BGP.IPvXAddress (NodeIP). Default behaviour is to listen
for BGP connections on all addresses.
type: string
communities:
description: Communities is a list of BGP community values and their
arbitrary names for tagging routes.
items:
description: Community contains standard or large community value
and its name.
properties:
name:
description: Name given to community value.
type: string
value:
description: Value must be of format `aa:nn` or `aa:nn:mm`.
For standard community use `aa:nn` format, where `aa` and
`nn` are 16 bit number. For large community use `aa:nn:mm`
format, where `aa`, `nn` and `mm` are 32 bit number. Where,
`aa` is an AS Number, `nn` and `mm` are per-AS identifier.
pattern: ^(\d+):(\d+)$|^(\d+):(\d+):(\d+)$
type: string
type: object
type: array
ignoredInterfaces:
description: IgnoredInterfaces indicates the network interfaces that
needs to be excluded when reading device routes.
items:
type: string
type: array
listenPort:
description: ListenPort is the port where BGP protocol should listen.
Defaults to 179
maximum: 65535
minimum: 1
type: integer
logSeverityScreen:
description: 'LogSeverityScreen is the log severity above which logs
are sent to the stdout. [Default: INFO]'
type: string
nodeMeshMaxRestartTime:
description: Time to allow for software restart for node-to-mesh peerings. When
specified, this is configured as the graceful restart timeout. When
not specified, the BIRD default of 120s is used. This field can
only be set on the default BGPConfiguration instance and requires
that NodeMesh is enabled
type: string
nodeMeshPassword:
description: Optional BGP password for full node-to-mesh peerings.
This field can only be set on the default BGPConfiguration instance
and requires that NodeMesh is enabled
properties:
secretKeyRef:
description: Selects a key of a secret in the node pod's namespace.
properties:
key:
description: The key of the secret to select from. Must be
a valid secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind, uid?'
type: string
optional:
description: Specify whether the Secret or its key must be
defined
type: boolean
required:
- key
type: object
type: object
nodeToNodeMeshEnabled:
description: 'NodeToNodeMeshEnabled sets whether full node to node
BGP mesh is enabled. [Default: true]'
type: boolean
prefixAdvertisements:
description: PrefixAdvertisements contains per-prefix advertisement
configuration.
items:
description: PrefixAdvertisement configures advertisement properties
for the specified CIDR.
properties:
cidr:
description: CIDR for which properties should be advertised.
type: string
communities:
description: Communities can be list of either community names
already defined in `Specs.Communities` or community value
of format `aa:nn` or `aa:nn:mm`. For standard community use
`aa:nn` format, where `aa` and `nn` are 16 bit number. For
large community use `aa:nn:mm` format, where `aa`, `nn` and
`mm` are 32 bit number. Where,`aa` is an AS Number, `nn` and
`mm` are per-AS identifier.
items:
type: string
type: array
type: object
type: array
serviceClusterIPs:
description: ServiceClusterIPs are the CIDR blocks from which service
cluster IPs are allocated. If specified, Calico will advertise these
blocks, as well as any cluster IPs within them.
items:
description: ServiceClusterIPBlock represents a single allowed ClusterIP
CIDR block.
properties:
cidr:
type: string
type: object
type: array
serviceExternalIPs:
description: ServiceExternalIPs are the CIDR blocks for Kubernetes
Service External IPs. Kubernetes Service ExternalIPs will only be
advertised if they are within one of these blocks.
items:
description: ServiceExternalIPBlock represents a single allowed
External IP CIDR block.
properties:
cidr:
type: string
type: object
type: array
serviceLoadBalancerIPs:
description: ServiceLoadBalancerIPs are the CIDR blocks for Kubernetes
Service LoadBalancer IPs. Kubernetes Service status.LoadBalancer.Ingress
IPs will only be advertised if they are within one of these blocks.
items:
description: ServiceLoadBalancerIPBlock represents a single allowed
LoadBalancer IP CIDR block.
properties:
cidr:
type: string
type: object
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: bgppeers.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BGPPeer
listKind: BGPPeerList
plural: bgppeers
singular: bgppeer
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: BGPPeerSpec contains the specification for a BGPPeer resource.
properties:
asNumber:
description: The AS Number of the peer.
format: int32
type: integer
keepOriginalNextHop:
description: Option to keep the original nexthop field when routes
are sent to a BGP Peer. Setting "true" configures the selected BGP
Peers node to use the "next hop keep;" instead of "next hop self;"(default)
in the specific branch of the Node on "bird.cfg".
type: boolean
maxRestartTime:
description: Time to allow for software restart. When specified,
this is configured as the graceful restart timeout. When not specified,
the BIRD default of 120s is used.
type: string
node:
description: The node name identifying the Calico node instance that
is targeted by this peer. If this is not set, and no nodeSelector
is specified, then this BGP peer selects all nodes in the cluster.
type: string
nodeSelector:
description: Selector for the nodes that should have this peering. When
this is set, the Node field must be empty.
type: string
numAllowedLocalASNumbers:
description: Maximum number of local AS numbers that are allowed in
the AS path for received routes. This removes BGP loop prevention
and should only be used if absolutely necesssary.
format: int32
type: integer
password:
description: Optional BGP password for the peerings generated by this
BGPPeer resource.
properties:
secretKeyRef:
description: Selects a key of a secret in the node pod's namespace.
properties:
key:
description: The key of the secret to select from. Must be
a valid secret key.
type: string
name:
description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
TODO: Add other useful fields. apiVersion, kind, uid?'
type: string
optional:
description: Specify whether the Secret or its key must be
defined
type: boolean
required:
- key
type: object
type: object
peerIP:
description: The IP address of the peer followed by an optional port
number to peer with. If port number is given, format should be `[]:port`
or `:` for IPv4. If optional port number is not set,
and this peer IP and ASNumber belongs to a calico/node with ListenPort
set in BGPConfiguration, then we use that port to peer.
type: string
peerSelector:
description: Selector for the remote nodes to peer with. When this
is set, the PeerIP and ASNumber fields must be empty. For each
peering between the local node and selected remote nodes, we configure
an IPv4 peering if both ends have NodeBGPSpec.IPv4Address specified,
and an IPv6 peering if both ends have NodeBGPSpec.IPv6Address specified. The
remote AS number comes from the remote node's NodeBGPSpec.ASNumber,
or the global default if that is not set.
type: string
reachableBy:
description: Add an exact, i.e. /32, static route toward peer IP in
order to prevent route flapping. ReachableBy contains the address
of the gateway which peer can be reached by.
type: string
sourceAddress:
description: Specifies whether and how to configure a source address
for the peerings generated by this BGPPeer resource. Default value
"UseNodeIP" means to configure the node IP as the source address. "None"
means not to configure a source address.
type: string
ttlSecurity:
description: TTLSecurity enables the generalized TTL security mechanism
(GTSM) which protects against spoofed packets by ignoring received
packets with a smaller than expected TTL value. The provided value
is the number of hops (edges) between the peers.
type: integer
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: blockaffinities.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BlockAffinity
listKind: BlockAffinityList
plural: blockaffinities
singular: blockaffinity
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: BlockAffinitySpec contains the specification for a BlockAffinity
resource.
properties:
cidr:
type: string
deleted:
description: Deleted indicates that this block affinity is being deleted.
This field is a string for compatibility with older releases that
mistakenly treat this field as a string.
type: string
node:
type: string
state:
type: string
required:
- cidr
- deleted
- node
- state
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: (devel)
creationTimestamp: null
name: caliconodestatuses.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: CalicoNodeStatus
listKind: CalicoNodeStatusList
plural: caliconodestatuses
singular: caliconodestatus
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: CalicoNodeStatusSpec contains the specification for a CalicoNodeStatus
resource.
properties:
classes:
description: Classes declares the types of information to monitor
for this calico/node, and allows for selective status reporting
about certain subsets of information.
items:
type: string
type: array
node:
description: The node name identifies the Calico node instance for
node status.
type: string
updatePeriodSeconds:
description: UpdatePeriodSeconds is the period at which CalicoNodeStatus
should be updated. Set to 0 to disable CalicoNodeStatus refresh.
Maximum update period is one day.
format: int32
type: integer
type: object
status:
description: CalicoNodeStatusStatus defines the observed state of CalicoNodeStatus.
No validation needed for status since it is updated by Calico.
properties:
agent:
description: Agent holds agent status on the node.
properties:
birdV4:
description: BIRDV4 represents the latest observed status of bird4.
properties:
lastBootTime:
description: LastBootTime holds the value of lastBootTime
from bird.ctl output.
type: string
lastReconfigurationTime:
description: LastReconfigurationTime holds the value of lastReconfigTime
from bird.ctl output.
type: string
routerID:
description: Router ID used by bird.
type: string
state:
description: The state of the BGP Daemon.
type: string
version:
description: Version of the BGP daemon
type: string
type: object
birdV6:
description: BIRDV6 represents the latest observed status of bird6.
properties:
lastBootTime:
description: LastBootTime holds the value of lastBootTime
from bird.ctl output.
type: string
lastReconfigurationTime:
description: LastReconfigurationTime holds the value of lastReconfigTime
from bird.ctl output.
type: string
routerID:
description: Router ID used by bird.
type: string
state:
description: The state of the BGP Daemon.
type: string
version:
description: Version of the BGP daemon
type: string
type: object
type: object
bgp:
description: BGP holds node BGP status.
properties:
numberEstablishedV4:
description: The total number of IPv4 established bgp sessions.
type: integer
numberEstablishedV6:
description: The total number of IPv6 established bgp sessions.
type: integer
numberNotEstablishedV4:
description: The total number of IPv4 non-established bgp sessions.
type: integer
numberNotEstablishedV6:
description: The total number of IPv6 non-established bgp sessions.
type: integer
peersV4:
description: PeersV4 represents IPv4 BGP peers status on the node.
items:
description: CalicoNodePeer contains the status of BGP peers
on the node.
properties:
peerIP:
description: IP address of the peer whose condition we are
reporting.
type: string
since:
description: Since the state or reason last changed.
type: string
state:
description: State is the BGP session state.
type: string
type:
description: Type indicates whether this peer is configured
via the node-to-node mesh, or via en explicit global or
per-node BGPPeer object.
type: string
type: object
type: array
peersV6:
description: PeersV6 represents IPv6 BGP peers status on the node.
items:
description: CalicoNodePeer contains the status of BGP peers
on the node.
properties:
peerIP:
description: IP address of the peer whose condition we are
reporting.
type: string
since:
description: Since the state or reason last changed.
type: string
state:
description: State is the BGP session state.
type: string
type:
description: Type indicates whether this peer is configured
via the node-to-node mesh, or via en explicit global or
per-node BGPPeer object.
type: string
type: object
type: array
required:
- numberEstablishedV4
- numberEstablishedV6
- numberNotEstablishedV4
- numberNotEstablishedV6
type: object
lastUpdated:
description: LastUpdated is a timestamp representing the server time
when CalicoNodeStatus object last updated. It is represented in
RFC3339 form and is in UTC.
format: date-time
nullable: true
type: string
routes:
description: Routes reports routes known to the Calico BGP daemon
on the node.
properties:
routesV4:
description: RoutesV4 represents IPv4 routes on the node.
items:
description: CalicoNodeRoute contains the status of BGP routes
on the node.
properties:
destination:
description: Destination of the route.
type: string
gateway:
description: Gateway for the destination.
type: string
interface:
description: Interface for the destination
type: string
learnedFrom:
description: LearnedFrom contains information regarding
where this route originated.
properties:
peerIP:
description: If sourceType is NodeMesh or BGPPeer, IP
address of the router that sent us this route.
type: string
sourceType:
description: Type of the source where a route is learned
from.
type: string
type: object
type:
description: Type indicates if the route is being used for
forwarding or not.
type: string
type: object
type: array
routesV6:
description: RoutesV6 represents IPv6 routes on the node.
items:
description: CalicoNodeRoute contains the status of BGP routes
on the node.
properties:
destination:
description: Destination of the route.
type: string
gateway:
description: Gateway for the destination.
type: string
interface:
description: Interface for the destination
type: string
learnedFrom:
description: LearnedFrom contains information regarding
where this route originated.
properties:
peerIP:
description: If sourceType is NodeMesh or BGPPeer, IP
address of the router that sent us this route.
type: string
sourceType:
description: Type of the source where a route is learned
from.
type: string
type: object
type:
description: Type indicates if the route is being used for
forwarding or not.
type: string
type: object
type: array
type: object
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: clusterinformations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: ClusterInformation
listKind: ClusterInformationList
plural: clusterinformations
singular: clusterinformation
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
description: ClusterInformation contains the cluster specific information.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ClusterInformationSpec contains the values of describing
the cluster.
properties:
calicoVersion:
description: CalicoVersion is the version of Calico that the cluster
is running
type: string
clusterGUID:
description: ClusterGUID is the GUID of the cluster
type: string
clusterType:
description: ClusterType describes the type of the cluster
type: string
datastoreReady:
description: DatastoreReady is used during significant datastore migrations
to signal to components such as Felix that it should wait before
accessing the datastore.
type: boolean
variant:
description: Variant declares which variant of Calico should be active.
type: string
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: felixconfigurations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: FelixConfiguration
listKind: FelixConfigurationList
plural: felixconfigurations
singular: felixconfiguration
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
description: Felix Configuration contains the configuration for Felix.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: FelixConfigurationSpec contains the values of the Felix configuration.
properties:
allowIPIPPacketsFromWorkloads:
description: 'AllowIPIPPacketsFromWorkloads controls whether Felix
will add a rule to drop IPIP encapsulated traffic from workloads
[Default: false]'
type: boolean
allowVXLANPacketsFromWorkloads:
description: 'AllowVXLANPacketsFromWorkloads controls whether Felix
will add a rule to drop VXLAN encapsulated traffic from workloads
[Default: false]'
type: boolean
awsSrcDstCheck:
description: 'Set source-destination-check on AWS EC2 instances. Accepted
value must be one of "DoNothing", "Enable" or "Disable". [Default:
DoNothing]'
enum:
- DoNothing
- Enable
- Disable
type: string
bpfConnectTimeLoadBalancingEnabled:
description: 'BPFConnectTimeLoadBalancingEnabled when in BPF mode,
controls whether Felix installs the connection-time load balancer. The
connect-time load balancer is required for the host to be able to
reach Kubernetes services and it improves the performance of pod-to-service
connections. The only reason to disable it is for debugging purposes. [Default:
true]'
type: boolean
bpfDataIfacePattern:
description: BPFDataIfacePattern is a regular expression that controls
which interfaces Felix should attach BPF programs to in order to
catch traffic to/from the network. This needs to match the interfaces
that Calico workload traffic flows over as well as any interfaces
that handle incoming traffic to nodeports and services from outside
the cluster. It should not match the workload interfaces (usually
named cali...).
type: string
bpfDisableUnprivileged:
description: 'BPFDisableUnprivileged, if enabled, Felix sets the kernel.unprivileged_bpf_disabled
sysctl to disable unprivileged use of BPF. This ensures that unprivileged
users cannot access Calico''s BPF maps and cannot insert their own
BPF programs to interfere with Calico''s. [Default: true]'
type: boolean
bpfEnabled:
description: 'BPFEnabled, if enabled Felix will use the BPF dataplane.
[Default: false]'
type: boolean
bpfEnforceRPF:
description: 'BPFEnforceRPF enforce strict RPF on all host interfaces
with BPF programs regardless of what is the per-interfaces or global
setting. Possible values are Disabled, Strict or Loose. [Default:
Strict]'
type: string
bpfExtToServiceConnmark:
description: 'BPFExtToServiceConnmark in BPF mode, control a 32bit
mark that is set on connections from an external client to a local
service. This mark allows us to control how packets of that connection
are routed within the host and how is routing interpreted by RPF
check. [Default: 0]'
type: integer
bpfExternalServiceMode:
description: 'BPFExternalServiceMode in BPF mode, controls how connections
from outside the cluster to services (node ports and cluster IPs)
are forwarded to remote workloads. If set to "Tunnel" then both
request and response traffic is tunneled to the remote node. If
set to "DSR", the request traffic is tunneled but the response traffic
is sent directly from the remote node. In "DSR" mode, the remote
node appears to use the IP of the ingress node; this requires a
permissive L2 network. [Default: Tunnel]'
type: string
bpfHostConntrackBypass:
description: 'BPFHostConntrackBypass Controls whether to bypass Linux
conntrack in BPF mode for workloads and services. [Default: true
- bypass Linux conntrack]'
type: boolean
bpfKubeProxyEndpointSlicesEnabled:
description: BPFKubeProxyEndpointSlicesEnabled in BPF mode, controls
whether Felix's embedded kube-proxy accepts EndpointSlices or not.
type: boolean
bpfKubeProxyIptablesCleanupEnabled:
description: 'BPFKubeProxyIptablesCleanupEnabled, if enabled in BPF
mode, Felix will proactively clean up the upstream Kubernetes kube-proxy''s
iptables chains. Should only be enabled if kube-proxy is not running. [Default:
true]'
type: boolean
bpfKubeProxyMinSyncPeriod:
description: 'BPFKubeProxyMinSyncPeriod, in BPF mode, controls the
minimum time between updates to the dataplane for Felix''s embedded
kube-proxy. Lower values give reduced set-up latency. Higher values
reduce Felix CPU usage by batching up more work. [Default: 1s]'
type: string
bpfL3IfacePattern:
description: BPFL3IfacePattern is a regular expression that allows
to list tunnel devices like wireguard or vxlan (i.e., L3 devices)
in addition to BPFDataIfacePattern. That is, tunnel interfaces not
created by Calico, that Calico workload traffic flows over as well
as any interfaces that handle incoming traffic to nodeports and
services from outside the cluster.
type: string
bpfLogLevel:
description: 'BPFLogLevel controls the log level of the BPF programs
when in BPF dataplane mode. One of "Off", "Info", or "Debug". The
logs are emitted to the BPF trace pipe, accessible with the command
`tc exec bpf debug`. [Default: Off].'
type: string
bpfMapSizeConntrack:
description: 'BPFMapSizeConntrack sets the size for the conntrack
map. This map must be large enough to hold an entry for each active
connection. Warning: changing the size of the conntrack map can
cause disruption.'
type: integer
bpfMapSizeIPSets:
description: BPFMapSizeIPSets sets the size for ipsets map. The IP
sets map must be large enough to hold an entry for each endpoint
matched by every selector in the source/destination matches in network
policy. Selectors such as "all()" can result in large numbers of
entries (one entry per endpoint in that case).
type: integer
bpfMapSizeIfState:
description: BPFMapSizeIfState sets the size for ifstate map. The
ifstate map must be large enough to hold an entry for each device
(host + workloads) on a host.
type: integer
bpfMapSizeNATAffinity:
type: integer
bpfMapSizeNATBackend:
description: BPFMapSizeNATBackend sets the size for nat back end map.
This is the total number of endpoints. This is mostly more than
the size of the number of services.
type: integer
bpfMapSizeNATFrontend:
description: BPFMapSizeNATFrontend sets the size for nat front end
map. FrontendMap should be large enough to hold an entry for each
nodeport, external IP and each port in each service.
type: integer
bpfMapSizeRoute:
description: BPFMapSizeRoute sets the size for the routes map. The
routes map should be large enough to hold one entry per workload
and a handful of entries per host (enough to cover its own IPs and
tunnel IPs).
type: integer
bpfPSNATPorts:
anyOf:
- type: integer
- type: string
description: 'BPFPSNATPorts sets the range from which we randomly
pick a port if there is a source port collision. This should be
within the ephemeral range as defined by RFC 6056 (1024–65535) and
preferably outside the ephemeral ranges used by common operating
systems. Linux uses 32768–60999, while others mostly use the IANA
defined range 49152–65535. It is not necessarily a problem if this
range overlaps with the operating systems. Both ends of the range
are inclusive. [Default: 20000:29999]'
pattern: ^.*
x-kubernetes-int-or-string: true
bpfPolicyDebugEnabled:
description: BPFPolicyDebugEnabled when true, Felix records detailed
information about the BPF policy programs, which can be examined
with the calico-bpf command-line tool.
type: boolean
chainInsertMode:
description: 'ChainInsertMode controls whether Felix hooks the kernel''s
top-level iptables chains by inserting a rule at the top of the
chain or by appending a rule at the bottom. insert is the safe default
since it prevents Calico''s rules from being bypassed. If you switch
to append mode, be sure that the other rules in the chains signal
acceptance by falling through to the Calico rules, otherwise the
Calico policy will be bypassed. [Default: insert]'
type: string
dataplaneDriver:
description: DataplaneDriver filename of the external dataplane driver
to use. Only used if UseInternalDataplaneDriver is set to false.
type: string
dataplaneWatchdogTimeout:
description: "DataplaneWatchdogTimeout is the readiness/liveness timeout
used for Felix's (internal) dataplane driver. Increase this value
if you experience spurious non-ready or non-live events when Felix
is under heavy load. Decrease the value to get felix to report non-live
or non-ready more quickly. [Default: 90s] \n Deprecated: replaced
by the generic HealthTimeoutOverrides."
type: string
debugDisableLogDropping:
type: boolean
debugMemoryProfilePath:
type: string
debugSimulateCalcGraphHangAfter:
type: string
debugSimulateDataplaneHangAfter:
type: string
defaultEndpointToHostAction:
description: 'DefaultEndpointToHostAction controls what happens to
traffic that goes from a workload endpoint to the host itself (after
the traffic hits the endpoint egress policy). By default Calico
blocks traffic from workload endpoints to the host itself with an
iptables "DROP" action. If you want to allow some or all traffic
from endpoint to host, set this parameter to RETURN or ACCEPT. Use
RETURN if you have your own rules in the iptables "INPUT" chain;
Calico will insert its rules at the top of that chain, then "RETURN"
packets to the "INPUT" chain once it has completed processing workload
endpoint egress policy. Use ACCEPT to unconditionally accept packets
from workloads after processing workload endpoint egress policy.
[Default: Drop]'
type: string
deviceRouteProtocol:
description: This defines the route protocol added to programmed device
routes, by default this will be RTPROT_BOOT when left blank.
type: integer
deviceRouteSourceAddress:
description: This is the IPv4 source address to use on programmed
device routes. By default the source address is left blank, leaving
the kernel to choose the source address used.
type: string
deviceRouteSourceAddressIPv6:
description: This is the IPv6 source address to use on programmed
device routes. By default the source address is left blank, leaving
the kernel to choose the source address used.
type: string
disableConntrackInvalidCheck:
type: boolean
endpointReportingDelay:
type: string
endpointReportingEnabled:
type: boolean
externalNodesList:
description: ExternalNodesCIDRList is a list of CIDR's of external-non-calico-nodes
which may source tunnel traffic and have the tunneled traffic be
accepted at calico nodes.
items:
type: string
type: array
failsafeInboundHostPorts:
description: 'FailsafeInboundHostPorts is a list of UDP/TCP ports
and CIDRs that Felix will allow incoming traffic to host endpoints
on irrespective of the security policy. This is useful to avoid
accidentally cutting off a host with incorrect configuration. For
back-compatibility, if the protocol is not specified, it defaults
to "tcp". If a CIDR is not specified, it will allow traffic from
all addresses. To disable all inbound host ports, use the value
none. The default value allows ssh access and DHCP. [Default: tcp:22,
udp:68, tcp:179, tcp:2379, tcp:2380, tcp:6443, tcp:6666, tcp:6667]'
items:
description: ProtoPort is combination of protocol, port, and CIDR.
Protocol and port must be specified.
properties:
net:
type: string
port:
type: integer
protocol:
type: string
required:
- port
- protocol
type: object
type: array
failsafeOutboundHostPorts:
description: 'FailsafeOutboundHostPorts is a list of UDP/TCP ports
and CIDRs that Felix will allow outgoing traffic from host endpoints
to irrespective of the security policy. This is useful to avoid
accidentally cutting off a host with incorrect configuration. For
back-compatibility, if the protocol is not specified, it defaults
to "tcp". If a CIDR is not specified, it will allow traffic from
all addresses. To disable all outbound host ports, use the value
none. The default value opens etcd''s standard ports to ensure that
Felix does not get cut off from etcd as well as allowing DHCP and
DNS. [Default: tcp:179, tcp:2379, tcp:2380, tcp:6443, tcp:6666,
tcp:6667, udp:53, udp:67]'
items:
description: ProtoPort is combination of protocol, port, and CIDR.
Protocol and port must be specified.
properties:
net:
type: string
port:
type: integer
protocol:
type: string
required:
- port
- protocol
type: object
type: array
featureDetectOverride:
description: FeatureDetectOverride is used to override feature detection
based on auto-detected platform capabilities. Values are specified
in a comma separated list with no spaces, example; "SNATFullyRandom=true,MASQFullyRandom=false,RestoreSupportsLock=". "true"
or "false" will force the feature, empty or omitted values are auto-detected.
type: string
featureGates:
description: FeatureGates is used to enable or disable tech-preview
Calico features. Values are specified in a comma separated list
with no spaces, example; "BPFConnectTimeLoadBalancingWorkaround=enabled,XyZ=false".
This is used to enable features that are not fully production ready.
type: string
floatingIPs:
description: FloatingIPs configures whether or not Felix will program
non-OpenStack floating IP addresses. (OpenStack-derived floating
IPs are always programmed, regardless of this setting.)
enum:
- Enabled
- Disabled
type: string
genericXDPEnabled:
description: 'GenericXDPEnabled enables Generic XDP so network cards
that don''t support XDP offload or driver modes can use XDP. This
is not recommended since it doesn''t provide better performance
than iptables. [Default: false]'
type: boolean
healthEnabled:
type: boolean
healthHost:
type: string
healthPort:
type: integer
healthTimeoutOverrides:
description: HealthTimeoutOverrides allows the internal watchdog timeouts
of individual subcomponents to be overriden. This is useful for
working around "false positive" liveness timeouts that can occur
in particularly stressful workloads or if CPU is constrained. For
a list of active subcomponents, see Felix's logs.
items:
properties:
name:
type: string
timeout:
type: string
required:
- name
- timeout
type: object
type: array
interfaceExclude:
description: 'InterfaceExclude is a comma-separated list of interfaces
that Felix should exclude when monitoring for host endpoints. The
default value ensures that Felix ignores Kubernetes'' IPVS dummy
interface, which is used internally by kube-proxy. If you want to
exclude multiple interface names using a single value, the list
supports regular expressions. For regular expressions you must wrap
the value with ''/''. For example having values ''/^kube/,veth1''
will exclude all interfaces that begin with ''kube'' and also the
interface ''veth1''. [Default: kube-ipvs0]'
type: string
interfacePrefix:
description: 'InterfacePrefix is the interface name prefix that identifies
workload endpoints and so distinguishes them from host endpoint
interfaces. Note: in environments other than bare metal, the orchestrators
configure this appropriately. For example our Kubernetes and Docker
integrations set the ''cali'' value, and our OpenStack integration
sets the ''tap'' value. [Default: cali]'
type: string
interfaceRefreshInterval:
description: InterfaceRefreshInterval is the period at which Felix
rescans local interfaces to verify their state. The rescan can be
disabled by setting the interval to 0.
type: string
ipipEnabled:
description: 'IPIPEnabled overrides whether Felix should configure
an IPIP interface on the host. Optional as Felix determines this
based on the existing IP pools. [Default: nil (unset)]'
type: boolean
ipipMTU:
description: 'IPIPMTU is the MTU to set on the tunnel device. See
Configuring MTU [Default: 1440]'
type: integer
ipsetsRefreshInterval:
description: 'IpsetsRefreshInterval is the period at which Felix re-checks
all iptables state to ensure that no other process has accidentally
broken Calico''s rules. Set to 0 to disable iptables refresh. [Default:
90s]'
type: string
iptablesBackend:
description: IptablesBackend specifies which backend of iptables will
be used. The default is Auto.
type: string
iptablesFilterAllowAction:
type: string
iptablesLockFilePath:
description: 'IptablesLockFilePath is the location of the iptables
lock file. You may need to change this if the lock file is not in
its standard location (for example if you have mapped it into Felix''s
container at a different path). [Default: /run/xtables.lock]'
type: string
iptablesLockProbeInterval:
description: 'IptablesLockProbeInterval is the time that Felix will
wait between attempts to acquire the iptables lock if it is not
available. Lower values make Felix more responsive when the lock
is contended, but use more CPU. [Default: 50ms]'
type: string
iptablesLockTimeout:
description: 'IptablesLockTimeout is the time that Felix will wait
for the iptables lock, or 0, to disable. To use this feature, Felix
must share the iptables lock file with all other processes that
also take the lock. When running Felix inside a container, this
requires the /run directory of the host to be mounted into the calico/node
or calico/felix container. [Default: 0s disabled]'
type: string
iptablesMangleAllowAction:
type: string
iptablesMarkMask:
description: 'IptablesMarkMask is the mask that Felix selects its
IPTables Mark bits from. Should be a 32 bit hexadecimal number with
at least 8 bits set, none of which clash with any other mark bits
in use on the system. [Default: 0xff000000]'
format: int32
type: integer
iptablesNATOutgoingInterfaceFilter:
type: string
iptablesPostWriteCheckInterval:
description: 'IptablesPostWriteCheckInterval is the period after Felix
has done a write to the dataplane that it schedules an extra read
back in order to check the write was not clobbered by another process.
This should only occur if another application on the system doesn''t
respect the iptables lock. [Default: 1s]'
type: string
iptablesRefreshInterval:
description: 'IptablesRefreshInterval is the period at which Felix
re-checks the IP sets in the dataplane to ensure that no other process
has accidentally broken Calico''s rules. Set to 0 to disable IP
sets refresh. Note: the default for this value is lower than the
other refresh intervals as a workaround for a Linux kernel bug that
was fixed in kernel version 4.11. If you are using v4.11 or greater
you may want to set this to, a higher value to reduce Felix CPU
usage. [Default: 10s]'
type: string
ipv6Support:
description: IPv6Support controls whether Felix enables support for
IPv6 (if supported by the in-use dataplane).
type: boolean
kubeNodePortRanges:
description: 'KubeNodePortRanges holds list of port ranges used for
service node ports. Only used if felix detects kube-proxy running
in ipvs mode. Felix uses these ranges to separate host and workload
traffic. [Default: 30000:32767].'
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
logDebugFilenameRegex:
description: LogDebugFilenameRegex controls which source code files
have their Debug log output included in the logs. Only logs from
files with names that match the given regular expression are included. The
filter only applies to Debug level logs.
type: string
logFilePath:
description: 'LogFilePath is the full path to the Felix log. Set to
none to disable file logging. [Default: /var/log/calico/felix.log]'
type: string
logPrefix:
description: 'LogPrefix is the log prefix that Felix uses when rendering
LOG rules. [Default: calico-packet]'
type: string
logSeverityFile:
description: 'LogSeverityFile is the log severity above which logs
are sent to the log file. [Default: Info]'
type: string
logSeverityScreen:
description: 'LogSeverityScreen is the log severity above which logs
are sent to the stdout. [Default: Info]'
type: string
logSeveritySys:
description: 'LogSeveritySys is the log severity above which logs
are sent to the syslog. Set to None for no logging to syslog. [Default:
Info]'
type: string
maxIpsetSize:
type: integer
metadataAddr:
description: 'MetadataAddr is the IP address or domain name of the
server that can answer VM queries for cloud-init metadata. In OpenStack,
this corresponds to the machine running nova-api (or in Ubuntu,
nova-api-metadata). A value of none (case insensitive) means that
Felix should not set up any NAT rule for the metadata path. [Default:
127.0.0.1]'
type: string
metadataPort:
description: 'MetadataPort is the port of the metadata server. This,
combined with global.MetadataAddr (if not ''None''), is used to
set up a NAT rule, from 169.254.169.254:80 to MetadataAddr:MetadataPort.
In most cases this should not need to be changed [Default: 8775].'
type: integer
mtuIfacePattern:
description: MTUIfacePattern is a regular expression that controls
which interfaces Felix should scan in order to calculate the host's
MTU. This should not match workload interfaces (usually named cali...).
type: string
natOutgoingAddress:
description: NATOutgoingAddress specifies an address to use when performing
source NAT for traffic in a natOutgoing pool that is leaving the
network. By default the address used is an address on the interface
the traffic is leaving on (ie it uses the iptables MASQUERADE target)
type: string
natPortRange:
anyOf:
- type: integer
- type: string
description: NATPortRange specifies the range of ports that is used
for port mapping when doing outgoing NAT. When unset the default
behavior of the network stack is used.
pattern: ^.*
x-kubernetes-int-or-string: true
netlinkTimeout:
type: string
openstackRegion:
description: 'OpenstackRegion is the name of the region that a particular
Felix belongs to. In a multi-region Calico/OpenStack deployment,
this must be configured somehow for each Felix (here in the datamodel,
or in felix.cfg or the environment on each compute node), and must
match the [calico] openstack_region value configured in neutron.conf
on each node. [Default: Empty]'
type: string
policySyncPathPrefix:
description: 'PolicySyncPathPrefix is used to by Felix to communicate
policy changes to external services, like Application layer policy.
[Default: Empty]'
type: string
prometheusGoMetricsEnabled:
description: 'PrometheusGoMetricsEnabled disables Go runtime metrics
collection, which the Prometheus client does by default, when set
to false. This reduces the number of metrics reported, reducing
Prometheus load. [Default: true]'
type: boolean
prometheusMetricsEnabled:
description: 'PrometheusMetricsEnabled enables the Prometheus metrics
server in Felix if set to true. [Default: false]'
type: boolean
prometheusMetricsHost:
description: 'PrometheusMetricsHost is the host that the Prometheus
metrics server should bind to. [Default: empty]'
type: string
prometheusMetricsPort:
description: 'PrometheusMetricsPort is the TCP port that the Prometheus
metrics server should bind to. [Default: 9091]'
type: integer
prometheusProcessMetricsEnabled:
description: 'PrometheusProcessMetricsEnabled disables process metrics
collection, which the Prometheus client does by default, when set
to false. This reduces the number of metrics reported, reducing
Prometheus load. [Default: true]'
type: boolean
prometheusWireGuardMetricsEnabled:
description: 'PrometheusWireGuardMetricsEnabled disables wireguard
metrics collection, which the Prometheus client does by default,
when set to false. This reduces the number of metrics reported,
reducing Prometheus load. [Default: true]'
type: boolean
removeExternalRoutes:
description: Whether or not to remove device routes that have not
been programmed by Felix. Disabling this will allow external applications
to also add device routes. This is enabled by default which means
we will remove externally added routes.
type: boolean
reportingInterval:
description: 'ReportingInterval is the interval at which Felix reports
its status into the datastore or 0 to disable. Must be non-zero
in OpenStack deployments. [Default: 30s]'
type: string
reportingTTL:
description: 'ReportingTTL is the time-to-live setting for process-wide
status reports. [Default: 90s]'
type: string
routeRefreshInterval:
description: 'RouteRefreshInterval is the period at which Felix re-checks
the routes in the dataplane to ensure that no other process has
accidentally broken Calico''s rules. Set to 0 to disable route refresh.
[Default: 90s]'
type: string
routeSource:
description: 'RouteSource configures where Felix gets its routing
information. - WorkloadIPs: use workload endpoints to construct
routes. - CalicoIPAM: the default - use IPAM data to construct routes.'
type: string
routeSyncDisabled:
description: RouteSyncDisabled will disable all operations performed
on the route table. Set to true to run in network-policy mode only.
type: boolean
routeTableRange:
description: Deprecated in favor of RouteTableRanges. Calico programs
additional Linux route tables for various purposes. RouteTableRange
specifies the indices of the route tables that Calico should use.
properties:
max:
type: integer
min:
type: integer
required:
- max
- min
type: object
routeTableRanges:
description: Calico programs additional Linux route tables for various
purposes. RouteTableRanges specifies a set of table index ranges
that Calico should use. Deprecates`RouteTableRange`, overrides `RouteTableRange`.
items:
properties:
max:
type: integer
min:
type: integer
required:
- max
- min
type: object
type: array
serviceLoopPrevention:
description: 'When service IP advertisement is enabled, prevent routing
loops to service IPs that are not in use, by dropping or rejecting
packets that do not get DNAT''d by kube-proxy. Unless set to "Disabled",
in which case such routing loops continue to be allowed. [Default:
Drop]'
type: string
sidecarAccelerationEnabled:
description: 'SidecarAccelerationEnabled enables experimental sidecar
acceleration [Default: false]'
type: boolean
usageReportingEnabled:
description: 'UsageReportingEnabled reports anonymous Calico version
number and cluster size to projectcalico.org. Logs warnings returned
by the usage server. For example, if a significant security vulnerability
has been discovered in the version of Calico being used. [Default:
true]'
type: boolean
usageReportingInitialDelay:
description: 'UsageReportingInitialDelay controls the minimum delay
before Felix makes a report. [Default: 300s]'
type: string
usageReportingInterval:
description: 'UsageReportingInterval controls the interval at which
Felix makes reports. [Default: 86400s]'
type: string
useInternalDataplaneDriver:
description: UseInternalDataplaneDriver, if true, Felix will use its
internal dataplane programming logic. If false, it will launch
an external dataplane driver and communicate with it over protobuf.
type: boolean
vxlanEnabled:
description: 'VXLANEnabled overrides whether Felix should create the
VXLAN tunnel device for IPv4 VXLAN networking. Optional as Felix
determines this based on the existing IP pools. [Default: nil (unset)]'
type: boolean
vxlanMTU:
description: 'VXLANMTU is the MTU to set on the IPv4 VXLAN tunnel
device. See Configuring MTU [Default: 1410]'
type: integer
vxlanMTUV6:
description: 'VXLANMTUV6 is the MTU to set on the IPv6 VXLAN tunnel
device. See Configuring MTU [Default: 1390]'
type: integer
vxlanPort:
type: integer
vxlanVNI:
type: integer
wireguardEnabled:
description: 'WireguardEnabled controls whether Wireguard is enabled
for IPv4 (encapsulating IPv4 traffic over an IPv4 underlay network).
[Default: false]'
type: boolean
wireguardEnabledV6:
description: 'WireguardEnabledV6 controls whether Wireguard is enabled
for IPv6 (encapsulating IPv6 traffic over an IPv6 underlay network).
[Default: false]'
type: boolean
wireguardHostEncryptionEnabled:
description: 'WireguardHostEncryptionEnabled controls whether Wireguard
host-to-host encryption is enabled. [Default: false]'
type: boolean
wireguardInterfaceName:
description: 'WireguardInterfaceName specifies the name to use for
the IPv4 Wireguard interface. [Default: wireguard.cali]'
type: string
wireguardInterfaceNameV6:
description: 'WireguardInterfaceNameV6 specifies the name to use for
the IPv6 Wireguard interface. [Default: wg-v6.cali]'
type: string
wireguardKeepAlive:
description: 'WireguardKeepAlive controls Wireguard PersistentKeepalive
option. Set 0 to disable. [Default: 0]'
type: string
wireguardListeningPort:
description: 'WireguardListeningPort controls the listening port used
by IPv4 Wireguard. [Default: 51820]'
type: integer
wireguardListeningPortV6:
description: 'WireguardListeningPortV6 controls the listening port
used by IPv6 Wireguard. [Default: 51821]'
type: integer
wireguardMTU:
description: 'WireguardMTU controls the MTU on the IPv4 Wireguard
interface. See Configuring MTU [Default: 1440]'
type: integer
wireguardMTUV6:
description: 'WireguardMTUV6 controls the MTU on the IPv6 Wireguard
interface. See Configuring MTU [Default: 1420]'
type: integer
wireguardRoutingRulePriority:
description: 'WireguardRoutingRulePriority controls the priority value
to use for the Wireguard routing rule. [Default: 99]'
type: integer
workloadSourceSpoofing:
description: WorkloadSourceSpoofing controls whether pods can use
the allowedSourcePrefixes annotation to send traffic with a source
IP address that is not theirs. This is disabled by default. When
set to "Any", pods can request any prefix.
type: string
xdpEnabled:
description: 'XDPEnabled enables XDP acceleration for suitable untracked
incoming deny rules. [Default: true]'
type: boolean
xdpRefreshInterval:
description: 'XDPRefreshInterval is the period at which Felix re-checks
all XDP state to ensure that no other process has accidentally broken
Calico''s BPF maps or attached programs. Set to 0 to disable XDP
refresh. [Default: 90s]'
type: string
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: globalnetworkpolicies.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: GlobalNetworkPolicy
listKind: GlobalNetworkPolicyList
plural: globalnetworkpolicies
singular: globalnetworkpolicy
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
applyOnForward:
description: ApplyOnForward indicates to apply the rules in this policy
on forward traffic.
type: boolean
doNotTrack:
description: DoNotTrack indicates whether packets matched by the rules
in this policy should go through the data plane's connection tracking,
such as Linux conntrack. If True, the rules in this policy are
applied before any data plane connection tracking, and packets allowed
by this policy are marked as not to be tracked.
type: boolean
egress:
description: The ordered set of egress rules. Each rule contains
a set of packet match criteria and a corresponding action to apply.
items:
description: "A Rule encapsulates a set of match criteria and an
action. Both selector-based security Policy and security Profiles
reference rules - separated out as a list of rules for both ingress
and egress packet matching. \n Each positive match criteria has
a negated version, prefixed with \"Not\". All the match criteria
within a rule must be satisfied for a packet to match. A single
rule can contain the positive and negative version of a match
and both must be satisfied for the rule to match."
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: "NamespaceSelector is an optional field that
contains a selector expression. Only traffic that originates
from (or terminates at) endpoints within the selected
namespaces will be matched. When both NamespaceSelector
and another selector are defined on the same rule, then
only workload endpoints that are matched by both selectors
will be selected by the rule. \n For NetworkPolicy, an
empty NamespaceSelector implies that the Selector is limited
to selecting only workload endpoints in the same namespace
as the NetworkPolicy. \n For NetworkPolicy, `global()`
NamespaceSelector implies that the Selector is limited
to selecting only GlobalNetworkSet or HostEndpoint. \n
For GlobalNetworkPolicy, an empty NamespaceSelector implies
the Selector applies to workload endpoints across all
namespaces."
type: string
nets:
description: Nets is an optional field that restricts the
rule to only apply to traffic that originates from (or
terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: NotPorts is the negated version of the Ports
field. Since only some protocols have ports, if any ports
are specified it requires the Protocol match in the Rule
to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: NotSelector is the negated version of the Selector
field. See Selector field for subtleties with negated
selectors.
type: string
ports:
description: "Ports is an optional field that restricts
the rule to only apply to traffic that has a source (destination)
port that matches one of these ranges/values. This value
is a list of integers or strings that represent ranges
of ports. \n Since only some protocols have ports, if
any ports are specified it requires the Protocol match
in the Rule to be set to \"TCP\" or \"UDP\"."
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for sample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching the selector will be matched. \n Note that: in
addition to the negated version of the Selector (see NotSelector
below), the selector expression syntax itself supports
negation. The two types of negation are subtly different.
One negates the set of matched endpoints, the other negates
the whole match: \n \tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled \tendpoints
that do not have the label \"my_label\". \n \tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled \tendpoints that do have the label \"my_label\".
\n The effect is that the latter will accept packets from
non-Calico sources whereas the former is limited to packets
from Calico-controlled endpoints."
type: string
serviceAccounts:
description: ServiceAccounts is an optional field that restricts
the rule to only apply to traffic that originates from
(or terminates at) a pod running as a matching service
account.
properties:
names:
description: Names is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account whose name is in the list.
items:
type: string
type: array
selector:
description: Selector is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account that matches the given label selector. If
both Names and Selector are specified then they are
AND'ed.
type: string
type: object
services:
description: "Services is an optional field that contains
options for matching Kubernetes Services. If specified,
only traffic that originates from or terminates at endpoints
within the selected service(s) will be matched, and only
to/from each endpoint's port. \n Services cannot be specified
on the same rule as Selector, NotSelector, NamespaceSelector,
Nets, NotNets or ServiceAccounts. \n Ports and NotPorts
can only be specified with Services on ingress rules."
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: Namespace specifies the namespace of the
given Service. If left empty, the rule will match
within this policy's namespace.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: Methods is an optional field that restricts
the rule to apply only to HTTP requests that use one of
the listed HTTP Methods (e.g. GET, PUT, etc.) Multiple
methods are OR'd together.
items:
type: string
type: array
paths:
description: 'Paths is an optional field that restricts
the rule to apply to HTTP requests that use one of the
listed HTTP Paths. Multiple paths are OR''d together.
e.g: - exact: /foo - prefix: /bar NOTE: Each entry may
ONLY specify either a `exact` or a `prefix` match. The
validator will check for it.'
items:
description: 'HTTPPath specifies an HTTP path to match.
It may be either of the form: exact: : which matches
the path exactly or prefix: : which matches
the path prefix'
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
description: ICMP is an optional field that restricts the rule
to apply to a specific type and code of ICMP traffic. This
should only be specified if the Protocol field is set to "ICMP"
or "ICMPv6".
properties:
code:
description: Match on a specific ICMP code. If specified,
the Type value must also be specified. This is a technical
limitation imposed by the kernel's iptables firewall,
which Calico uses to enforce the rule.
type: integer
type:
description: Match on a specific ICMP type. For example
a value of 8 refers to ICMP Echo Request (i.e. pings).
type: integer
type: object
ipVersion:
description: IPVersion is an optional field that restricts the
rule to only match a specific IP version.
type: integer
metadata:
description: Metadata contains additional information for this
rule
properties:
annotations:
additionalProperties:
type: string
description: Annotations is a set of key value pairs that
give extra information about the rule
type: object
type: object
notICMP:
description: NotICMP is the negated version of the ICMP field.
properties:
code:
description: Match on a specific ICMP code. If specified,
the Type value must also be specified. This is a technical
limitation imposed by the kernel's iptables firewall,
which Calico uses to enforce the rule.
type: integer
type:
description: Match on a specific ICMP type. For example
a value of 8 refers to ICMP Echo Request (i.e. pings).
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
description: NotProtocol is the negated version of the Protocol
field.
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
description: "Protocol is an optional field that restricts the
rule to only apply to traffic of a specific IP protocol. Required
if any of the EntityRules contain Ports (because ports only
apply to certain protocols). \n Must be one of these string
values: \"TCP\", \"UDP\", \"ICMP\", \"ICMPv6\", \"SCTP\",
\"UDPLite\" or an integer in the range 1-255."
pattern: ^.*
x-kubernetes-int-or-string: true
source:
description: Source contains the match criteria that apply to
source entity.
properties:
namespaceSelector:
description: "NamespaceSelector is an optional field that
contains a selector expression. Only traffic that originates
from (or terminates at) endpoints within the selected
namespaces will be matched. When both NamespaceSelector
and another selector are defined on the same rule, then
only workload endpoints that are matched by both selectors
will be selected by the rule. \n For NetworkPolicy, an
empty NamespaceSelector implies that the Selector is limited
to selecting only workload endpoints in the same namespace
as the NetworkPolicy. \n For NetworkPolicy, `global()`
NamespaceSelector implies that the Selector is limited
to selecting only GlobalNetworkSet or HostEndpoint. \n
For GlobalNetworkPolicy, an empty NamespaceSelector implies
the Selector applies to workload endpoints across all
namespaces."
type: string
nets:
description: Nets is an optional field that restricts the
rule to only apply to traffic that originates from (or
terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: NotPorts is the negated version of the Ports
field. Since only some protocols have ports, if any ports
are specified it requires the Protocol match in the Rule
to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: NotSelector is the negated version of the Selector
field. See Selector field for subtleties with negated
selectors.
type: string
ports:
description: "Ports is an optional field that restricts
the rule to only apply to traffic that has a source (destination)
port that matches one of these ranges/values. This value
is a list of integers or strings that represent ranges
of ports. \n Since only some protocols have ports, if
any ports are specified it requires the Protocol match
in the Rule to be set to \"TCP\" or \"UDP\"."
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for sample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching the selector will be matched. \n Note that: in
addition to the negated version of the Selector (see NotSelector
below), the selector expression syntax itself supports
negation. The two types of negation are subtly different.
One negates the set of matched endpoints, the other negates
the whole match: \n \tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled \tendpoints
that do not have the label \"my_label\". \n \tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled \tendpoints that do have the label \"my_label\".
\n The effect is that the latter will accept packets from
non-Calico sources whereas the former is limited to packets
from Calico-controlled endpoints."
type: string
serviceAccounts:
description: ServiceAccounts is an optional field that restricts
the rule to only apply to traffic that originates from
(or terminates at) a pod running as a matching service
account.
properties:
names:
description: Names is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account whose name is in the list.
items:
type: string
type: array
selector:
description: Selector is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account that matches the given label selector. If
both Names and Selector are specified then they are
AND'ed.
type: string
type: object
services:
description: "Services is an optional field that contains
options for matching Kubernetes Services. If specified,
only traffic that originates from or terminates at endpoints
within the selected service(s) will be matched, and only
to/from each endpoint's port. \n Services cannot be specified
on the same rule as Selector, NotSelector, NamespaceSelector,
Nets, NotNets or ServiceAccounts. \n Ports and NotPorts
can only be specified with Services on ingress rules."
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: Namespace specifies the namespace of the
given Service. If left empty, the rule will match
within this policy's namespace.
type: string
type: object
type: object
required:
- action
type: object
type: array
ingress:
description: The ordered set of ingress rules. Each rule contains
a set of packet match criteria and a corresponding action to apply.
items:
description: "A Rule encapsulates a set of match criteria and an
action. Both selector-based security Policy and security Profiles
reference rules - separated out as a list of rules for both ingress
and egress packet matching. \n Each positive match criteria has
a negated version, prefixed with \"Not\". All the match criteria
within a rule must be satisfied for a packet to match. A single
rule can contain the positive and negative version of a match
and both must be satisfied for the rule to match."
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: "NamespaceSelector is an optional field that
contains a selector expression. Only traffic that originates
from (or terminates at) endpoints within the selected
namespaces will be matched. When both NamespaceSelector
and another selector are defined on the same rule, then
only workload endpoints that are matched by both selectors
will be selected by the rule. \n For NetworkPolicy, an
empty NamespaceSelector implies that the Selector is limited
to selecting only workload endpoints in the same namespace
as the NetworkPolicy. \n For NetworkPolicy, `global()`
NamespaceSelector implies that the Selector is limited
to selecting only GlobalNetworkSet or HostEndpoint. \n
For GlobalNetworkPolicy, an empty NamespaceSelector implies
the Selector applies to workload endpoints across all
namespaces."
type: string
nets:
description: Nets is an optional field that restricts the
rule to only apply to traffic that originates from (or
terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: NotPorts is the negated version of the Ports
field. Since only some protocols have ports, if any ports
are specified it requires the Protocol match in the Rule
to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: NotSelector is the negated version of the Selector
field. See Selector field for subtleties with negated
selectors.
type: string
ports:
description: "Ports is an optional field that restricts
the rule to only apply to traffic that has a source (destination)
port that matches one of these ranges/values. This value
is a list of integers or strings that represent ranges
of ports. \n Since only some protocols have ports, if
any ports are specified it requires the Protocol match
in the Rule to be set to \"TCP\" or \"UDP\"."
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for sample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching the selector will be matched. \n Note that: in
addition to the negated version of the Selector (see NotSelector
below), the selector expression syntax itself supports
negation. The two types of negation are subtly different.
One negates the set of matched endpoints, the other negates
the whole match: \n \tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled \tendpoints
that do not have the label \"my_label\". \n \tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled \tendpoints that do have the label \"my_label\".
\n The effect is that the latter will accept packets from
non-Calico sources whereas the former is limited to packets
from Calico-controlled endpoints."
type: string
serviceAccounts:
description: ServiceAccounts is an optional field that restricts
the rule to only apply to traffic that originates from
(or terminates at) a pod running as a matching service
account.
properties:
names:
description: Names is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account whose name is in the list.
items:
type: string
type: array
selector:
description: Selector is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account that matches the given label selector. If
both Names and Selector are specified then they are
AND'ed.
type: string
type: object
services:
description: "Services is an optional field that contains
options for matching Kubernetes Services. If specified,
only traffic that originates from or terminates at endpoints
within the selected service(s) will be matched, and only
to/from each endpoint's port. \n Services cannot be specified
on the same rule as Selector, NotSelector, NamespaceSelector,
Nets, NotNets or ServiceAccounts. \n Ports and NotPorts
can only be specified with Services on ingress rules."
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: Namespace specifies the namespace of the
given Service. If left empty, the rule will match
within this policy's namespace.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: Methods is an optional field that restricts
the rule to apply only to HTTP requests that use one of
the listed HTTP Methods (e.g. GET, PUT, etc.) Multiple
methods are OR'd together.
items:
type: string
type: array
paths:
description: 'Paths is an optional field that restricts
the rule to apply to HTTP requests that use one of the
listed HTTP Paths. Multiple paths are OR''d together.
e.g: - exact: /foo - prefix: /bar NOTE: Each entry may
ONLY specify either a `exact` or a `prefix` match. The
validator will check for it.'
items:
description: 'HTTPPath specifies an HTTP path to match.
It may be either of the form: exact: : which matches
the path exactly or prefix: : which matches
the path prefix'
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
description: ICMP is an optional field that restricts the rule
to apply to a specific type and code of ICMP traffic. This
should only be specified if the Protocol field is set to "ICMP"
or "ICMPv6".
properties:
code:
description: Match on a specific ICMP code. If specified,
the Type value must also be specified. This is a technical
limitation imposed by the kernel's iptables firewall,
which Calico uses to enforce the rule.
type: integer
type:
description: Match on a specific ICMP type. For example
a value of 8 refers to ICMP Echo Request (i.e. pings).
type: integer
type: object
ipVersion:
description: IPVersion is an optional field that restricts the
rule to only match a specific IP version.
type: integer
metadata:
description: Metadata contains additional information for this
rule
properties:
annotations:
additionalProperties:
type: string
description: Annotations is a set of key value pairs that
give extra information about the rule
type: object
type: object
notICMP:
description: NotICMP is the negated version of the ICMP field.
properties:
code:
description: Match on a specific ICMP code. If specified,
the Type value must also be specified. This is a technical
limitation imposed by the kernel's iptables firewall,
which Calico uses to enforce the rule.
type: integer
type:
description: Match on a specific ICMP type. For example
a value of 8 refers to ICMP Echo Request (i.e. pings).
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
description: NotProtocol is the negated version of the Protocol
field.
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
description: "Protocol is an optional field that restricts the
rule to only apply to traffic of a specific IP protocol. Required
if any of the EntityRules contain Ports (because ports only
apply to certain protocols). \n Must be one of these string
values: \"TCP\", \"UDP\", \"ICMP\", \"ICMPv6\", \"SCTP\",
\"UDPLite\" or an integer in the range 1-255."
pattern: ^.*
x-kubernetes-int-or-string: true
source:
description: Source contains the match criteria that apply to
source entity.
properties:
namespaceSelector:
description: "NamespaceSelector is an optional field that
contains a selector expression. Only traffic that originates
from (or terminates at) endpoints within the selected
namespaces will be matched. When both NamespaceSelector
and another selector are defined on the same rule, then
only workload endpoints that are matched by both selectors
will be selected by the rule. \n For NetworkPolicy, an
empty NamespaceSelector implies that the Selector is limited
to selecting only workload endpoints in the same namespace
as the NetworkPolicy. \n For NetworkPolicy, `global()`
NamespaceSelector implies that the Selector is limited
to selecting only GlobalNetworkSet or HostEndpoint. \n
For GlobalNetworkPolicy, an empty NamespaceSelector implies
the Selector applies to workload endpoints across all
namespaces."
type: string
nets:
description: Nets is an optional field that restricts the
rule to only apply to traffic that originates from (or
terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: NotPorts is the negated version of the Ports
field. Since only some protocols have ports, if any ports
are specified it requires the Protocol match in the Rule
to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: NotSelector is the negated version of the Selector
field. See Selector field for subtleties with negated
selectors.
type: string
ports:
description: "Ports is an optional field that restricts
the rule to only apply to traffic that has a source (destination)
port that matches one of these ranges/values. This value
is a list of integers or strings that represent ranges
of ports. \n Since only some protocols have ports, if
any ports are specified it requires the Protocol match
in the Rule to be set to \"TCP\" or \"UDP\"."
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for sample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching the selector will be matched. \n Note that: in
addition to the negated version of the Selector (see NotSelector
below), the selector expression syntax itself supports
negation. The two types of negation are subtly different.
One negates the set of matched endpoints, the other negates
the whole match: \n \tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled \tendpoints
that do not have the label \"my_label\". \n \tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled \tendpoints that do have the label \"my_label\".
\n The effect is that the latter will accept packets from
non-Calico sources whereas the former is limited to packets
from Calico-controlled endpoints."
type: string
serviceAccounts:
description: ServiceAccounts is an optional field that restricts
the rule to only apply to traffic that originates from
(or terminates at) a pod running as a matching service
account.
properties:
names:
description: Names is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account whose name is in the list.
items:
type: string
type: array
selector:
description: Selector is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account that matches the given label selector. If
both Names and Selector are specified then they are
AND'ed.
type: string
type: object
services:
description: "Services is an optional field that contains
options for matching Kubernetes Services. If specified,
only traffic that originates from or terminates at endpoints
within the selected service(s) will be matched, and only
to/from each endpoint's port. \n Services cannot be specified
on the same rule as Selector, NotSelector, NamespaceSelector,
Nets, NotNets or ServiceAccounts. \n Ports and NotPorts
can only be specified with Services on ingress rules."
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: Namespace specifies the namespace of the
given Service. If left empty, the rule will match
within this policy's namespace.
type: string
type: object
type: object
required:
- action
type: object
type: array
namespaceSelector:
description: NamespaceSelector is an optional field for an expression
used to select a pod based on namespaces.
type: string
order:
description: Order is an optional field that specifies the order in
which the policy is applied. Policies with higher "order" are applied
after those with lower order. If the order is omitted, it may be
considered to be "infinite" - i.e. the policy will be applied last. Policies
with identical order will be applied in alphanumerical order based
on the Policy "Name".
type: number
preDNAT:
description: PreDNAT indicates to apply the rules in this policy before
any DNAT.
type: boolean
selector:
description: "The selector is an expression used to pick pick out
the endpoints that the policy should be applied to. \n Selector
expressions follow this syntax: \n \tlabel == \"string_literal\"
\ -> comparison, e.g. my_label == \"foo bar\" \tlabel != \"string_literal\"
\ -> not equal; also matches if label is not present \tlabel in
{ \"a\", \"b\", \"c\", ... } -> true if the value of label X is
one of \"a\", \"b\", \"c\" \tlabel not in { \"a\", \"b\", \"c\",
... } -> true if the value of label X is not one of \"a\", \"b\",
\"c\" \thas(label_name) -> True if that label is present \t! expr
-> negation of expr \texpr && expr -> Short-circuit and \texpr
|| expr -> Short-circuit or \t( expr ) -> parens for grouping \tall()
or the empty selector -> matches all endpoints. \n Label names are
allowed to contain alphanumerics, -, _ and /. String literals are
more permissive but they do not support escape characters. \n Examples
(with made-up labels): \n \ttype == \"webserver\" && deployment
== \"prod\" \ttype in {\"frontend\", \"backend\"} \tdeployment !=
\"dev\" \t! has(label_name)"
type: string
serviceAccountSelector:
description: ServiceAccountSelector is an optional field for an expression
used to select a pod based on service accounts.
type: string
types:
description: "Types indicates whether this policy applies to ingress,
or to egress, or to both. When not explicitly specified (and so
the value on creation is empty or nil), Calico defaults Types according
to what Ingress and Egress rules are present in the policy. The
default is: \n - [ PolicyTypeIngress ], if there are no Egress rules
(including the case where there are also no Ingress rules) \n
- [ PolicyTypeEgress ], if there are Egress rules but no Ingress
rules \n - [ PolicyTypeIngress, PolicyTypeEgress ], if there are
both Ingress and Egress rules. \n When the policy is read back again,
Types will always be one of these values, never empty or nil."
items:
description: PolicyType enumerates the possible values of the PolicySpec
Types field.
type: string
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: globalnetworksets.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: GlobalNetworkSet
listKind: GlobalNetworkSetList
plural: globalnetworksets
singular: globalnetworkset
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
description: GlobalNetworkSet contains a set of arbitrary IP sub-networks/CIDRs
that share labels to allow rules to refer to them via selectors. The labels
of GlobalNetworkSet are not namespaced.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: GlobalNetworkSetSpec contains the specification for a NetworkSet
resource.
properties:
nets:
description: The list of IP networks that belong to this set.
items:
type: string
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: hostendpoints.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: HostEndpoint
listKind: HostEndpointList
plural: hostendpoints
singular: hostendpoint
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: HostEndpointSpec contains the specification for a HostEndpoint
resource.
properties:
expectedIPs:
description: "The expected IP addresses (IPv4 and IPv6) of the endpoint.
If \"InterfaceName\" is not present, Calico will look for an interface
matching any of the IPs in the list and apply policy to that. Note:
\tWhen using the selector match criteria in an ingress or egress
security Policy \tor Profile, Calico converts the selector into
a set of IP addresses. For host \tendpoints, the ExpectedIPs field
is used for that purpose. (If only the interface \tname is specified,
Calico does not learn the IPs of the interface for use in match
\tcriteria.)"
items:
type: string
type: array
interfaceName:
description: "Either \"*\", or the name of a specific Linux interface
to apply policy to; or empty. \"*\" indicates that this HostEndpoint
governs all traffic to, from or through the default network namespace
of the host named by the \"Node\" field; entering and leaving that
namespace via any interface, including those from/to non-host-networked
local workloads. \n If InterfaceName is not \"*\", this HostEndpoint
only governs traffic that enters or leaves the host through the
specific interface named by InterfaceName, or - when InterfaceName
is empty - through the specific interface that has one of the IPs
in ExpectedIPs. Therefore, when InterfaceName is empty, at least
one expected IP must be specified. Only external interfaces (such
as \"eth0\") are supported here; it isn't possible for a HostEndpoint
to protect traffic through a specific local workload interface.
\n Note: Only some kinds of policy are implemented for \"*\" HostEndpoints;
initially just pre-DNAT policy. Please check Calico documentation
for the latest position."
type: string
node:
description: The node name identifying the Calico node instance.
type: string
ports:
description: Ports contains the endpoint's named ports, which may
be referenced in security policy rules.
items:
properties:
name:
type: string
port:
type: integer
protocol:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
required:
- name
- port
- protocol
type: object
type: array
profiles:
description: A list of identifiers of security Profile objects that
apply to this endpoint. Each profile is applied in the order that
they appear in this list. Profile rules are applied after the selector-based
security policy.
items:
type: string
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: ipamblocks.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPAMBlock
listKind: IPAMBlockList
plural: ipamblocks
singular: ipamblock
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: IPAMBlockSpec contains the specification for an IPAMBlock
resource.
properties:
affinity:
description: Affinity of the block, if this block has one. If set,
it will be of the form "host:". If not set, this block
is not affine to a host.
type: string
allocations:
description: Array of allocations in-use within this block. nil entries
mean the allocation is free. For non-nil entries at index i, the
index is the ordinal of the allocation within this block and the
value is the index of the associated attributes in the Attributes
array.
items:
type: integer
# TODO: This nullable is manually added in. We should update controller-gen
# to handle []*int properly itself.
nullable: true
type: array
attributes:
description: Attributes is an array of arbitrary metadata associated
with allocations in the block. To find attributes for a given allocation,
use the value of the allocation's entry in the Allocations array
as the index of the element in this array.
items:
properties:
handle_id:
type: string
secondary:
additionalProperties:
type: string
type: object
type: object
type: array
cidr:
description: The block's CIDR.
type: string
deleted:
description: Deleted is an internal boolean used to workaround a limitation
in the Kubernetes API whereby deletion will not return a conflict
error if the block has been updated. It should not be set manually.
type: boolean
sequenceNumber:
default: 0
description: We store a sequence number that is updated each time
the block is written. Each allocation will also store the sequence
number of the block at the time of its creation. When releasing
an IP, passing the sequence number associated with the allocation
allows us to protect against a race condition and ensure the IP
hasn't been released and re-allocated since the release request.
format: int64
type: integer
sequenceNumberForAllocation:
additionalProperties:
format: int64
type: integer
description: Map of allocated ordinal within the block to sequence
number of the block at the time of allocation. Kubernetes does not
allow numerical keys for maps, so the key is cast to a string.
type: object
strictAffinity:
description: StrictAffinity on the IPAMBlock is deprecated and no
longer used by the code. Use IPAMConfig StrictAffinity instead.
type: boolean
unallocated:
description: Unallocated is an ordered list of allocations which are
free in the block.
items:
type: integer
type: array
required:
- allocations
- attributes
- cidr
- strictAffinity
- unallocated
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: ipamconfigs.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPAMConfig
listKind: IPAMConfigList
plural: ipamconfigs
singular: ipamconfig
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: IPAMConfigSpec contains the specification for an IPAMConfig
resource.
properties:
autoAllocateBlocks:
type: boolean
maxBlocksPerHost:
description: MaxBlocksPerHost, if non-zero, is the max number of blocks
that can be affine to each host.
maximum: 2147483647
minimum: 0
type: integer
strictAffinity:
type: boolean
required:
- autoAllocateBlocks
- strictAffinity
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: ipamhandles.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPAMHandle
listKind: IPAMHandleList
plural: ipamhandles
singular: ipamhandle
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: IPAMHandleSpec contains the specification for an IPAMHandle
resource.
properties:
block:
additionalProperties:
type: integer
type: object
deleted:
type: boolean
handleID:
type: string
required:
- block
- handleID
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: ippools.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPPool
listKind: IPPoolList
plural: ippools
singular: ippool
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: IPPoolSpec contains the specification for an IPPool resource.
properties:
allowedUses:
description: AllowedUse controls what the IP pool will be used for. If
not specified or empty, defaults to ["Tunnel", "Workload"] for back-compatibility
items:
type: string
type: array
blockSize:
description: The block size to use for IP address assignments from
this pool. Defaults to 26 for IPv4 and 122 for IPv6.
type: integer
cidr:
description: The pool CIDR.
type: string
disableBGPExport:
description: 'Disable exporting routes from this IP Pool''s CIDR over
BGP. [Default: false]'
type: boolean
disabled:
description: When disabled is true, Calico IPAM will not assign addresses
from this pool.
type: boolean
ipip:
description: 'Deprecated: this field is only used for APIv1 backwards
compatibility. Setting this field is not allowed, this field is
for internal use only.'
properties:
enabled:
description: When enabled is true, ipip tunneling will be used
to deliver packets to destinations within this pool.
type: boolean
mode:
description: The IPIP mode. This can be one of "always" or "cross-subnet". A
mode of "always" will also use IPIP tunneling for routing to
destination IP addresses within this pool. A mode of "cross-subnet"
will only use IPIP tunneling when the destination node is on
a different subnet to the originating node. The default value
(if not specified) is "always".
type: string
type: object
ipipMode:
description: Contains configuration for IPIP tunneling for this pool.
If not specified, then this is defaulted to "Never" (i.e. IPIP tunneling
is disabled).
type: string
nat-outgoing:
description: 'Deprecated: this field is only used for APIv1 backwards
compatibility. Setting this field is not allowed, this field is
for internal use only.'
type: boolean
natOutgoing:
description: When natOutgoing is true, packets sent from Calico networked
containers in this pool to destinations outside of this pool will
be masqueraded.
type: boolean
nodeSelector:
description: Allows IPPool to allocate for a specific node by label
selector.
type: string
vxlanMode:
description: Contains configuration for VXLAN tunneling for this pool.
If not specified, then this is defaulted to "Never" (i.e. VXLAN
tunneling is disabled).
type: string
required:
- cidr
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: (devel)
creationTimestamp: null
name: ipreservations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPReservation
listKind: IPReservationList
plural: ipreservations
singular: ipreservation
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: IPReservationSpec contains the specification for an IPReservation
resource.
properties:
reservedCIDRs:
description: ReservedCIDRs is a list of CIDRs and/or IP addresses
that Calico IPAM will exclude from new allocations.
items:
type: string
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: kubecontrollersconfigurations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: KubeControllersConfiguration
listKind: KubeControllersConfigurationList
plural: kubecontrollersconfigurations
singular: kubecontrollersconfiguration
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: KubeControllersConfigurationSpec contains the values of the
Kubernetes controllers configuration.
properties:
controllers:
description: Controllers enables and configures individual Kubernetes
controllers
properties:
namespace:
description: Namespace enables and configures the namespace controller.
Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform reconciliation
with the Calico datastore. [Default: 5m]'
type: string
type: object
node:
description: Node enables and configures the node controller.
Enabled by default, set to nil to disable.
properties:
hostEndpoint:
description: HostEndpoint controls syncing nodes to host endpoints.
Disabled by default, set to nil to disable.
properties:
autoCreate:
description: 'AutoCreate enables automatic creation of
host endpoints for every node. [Default: Disabled]'
type: string
type: object
leakGracePeriod:
description: 'LeakGracePeriod is the period used by the controller
to determine if an IP address has been leaked. Set to 0
to disable IP garbage collection. [Default: 15m]'
type: string
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform reconciliation
with the Calico datastore. [Default: 5m]'
type: string
syncLabels:
description: 'SyncLabels controls whether to copy Kubernetes
node labels to Calico nodes. [Default: Enabled]'
type: string
type: object
policy:
description: Policy enables and configures the policy controller.
Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform reconciliation
with the Calico datastore. [Default: 5m]'
type: string
type: object
serviceAccount:
description: ServiceAccount enables and configures the service
account controller. Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform reconciliation
with the Calico datastore. [Default: 5m]'
type: string
type: object
workloadEndpoint:
description: WorkloadEndpoint enables and configures the workload
endpoint controller. Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform reconciliation
with the Calico datastore. [Default: 5m]'
type: string
type: object
type: object
debugProfilePort:
description: DebugProfilePort configures the port to serve memory
and cpu profiles on. If not specified, profiling is disabled.
format: int32
type: integer
etcdV3CompactionPeriod:
description: 'EtcdV3CompactionPeriod is the period between etcdv3
compaction requests. Set to 0 to disable. [Default: 10m]'
type: string
healthChecks:
description: 'HealthChecks enables or disables support for health
checks [Default: Enabled]'
type: string
logSeverityScreen:
description: 'LogSeverityScreen is the log severity above which logs
are sent to the stdout. [Default: Info]'
type: string
prometheusMetricsPort:
description: 'PrometheusMetricsPort is the TCP port that the Prometheus
metrics server should bind to. Set to 0 to disable. [Default: 9094]'
type: integer
required:
- controllers
type: object
status:
description: KubeControllersConfigurationStatus represents the status
of the configuration. It's useful for admins to be able to see the actual
config that was applied, which can be modified by environment variables
on the kube-controllers process.
properties:
environmentVars:
additionalProperties:
type: string
description: EnvironmentVars contains the environment variables on
the kube-controllers that influenced the RunningConfig.
type: object
runningConfig:
description: RunningConfig contains the effective config that is running
in the kube-controllers pod, after merging the API resource with
any environment variables.
properties:
controllers:
description: Controllers enables and configures individual Kubernetes
controllers
properties:
namespace:
description: Namespace enables and configures the namespace
controller. Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform
reconciliation with the Calico datastore. [Default:
5m]'
type: string
type: object
node:
description: Node enables and configures the node controller.
Enabled by default, set to nil to disable.
properties:
hostEndpoint:
description: HostEndpoint controls syncing nodes to host
endpoints. Disabled by default, set to nil to disable.
properties:
autoCreate:
description: 'AutoCreate enables automatic creation
of host endpoints for every node. [Default: Disabled]'
type: string
type: object
leakGracePeriod:
description: 'LeakGracePeriod is the period used by the
controller to determine if an IP address has been leaked.
Set to 0 to disable IP garbage collection. [Default:
15m]'
type: string
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform
reconciliation with the Calico datastore. [Default:
5m]'
type: string
syncLabels:
description: 'SyncLabels controls whether to copy Kubernetes
node labels to Calico nodes. [Default: Enabled]'
type: string
type: object
policy:
description: Policy enables and configures the policy controller.
Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform
reconciliation with the Calico datastore. [Default:
5m]'
type: string
type: object
serviceAccount:
description: ServiceAccount enables and configures the service
account controller. Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform
reconciliation with the Calico datastore. [Default:
5m]'
type: string
type: object
workloadEndpoint:
description: WorkloadEndpoint enables and configures the workload
endpoint controller. Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform
reconciliation with the Calico datastore. [Default:
5m]'
type: string
type: object
type: object
debugProfilePort:
description: DebugProfilePort configures the port to serve memory
and cpu profiles on. If not specified, profiling is disabled.
format: int32
type: integer
etcdV3CompactionPeriod:
description: 'EtcdV3CompactionPeriod is the period between etcdv3
compaction requests. Set to 0 to disable. [Default: 10m]'
type: string
healthChecks:
description: 'HealthChecks enables or disables support for health
checks [Default: Enabled]'
type: string
logSeverityScreen:
description: 'LogSeverityScreen is the log severity above which
logs are sent to the stdout. [Default: Info]'
type: string
prometheusMetricsPort:
description: 'PrometheusMetricsPort is the TCP port that the Prometheus
metrics server should bind to. Set to 0 to disable. [Default:
9094]'
type: integer
required:
- controllers
type: object
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: networkpolicies.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: NetworkPolicy
listKind: NetworkPolicyList
plural: networkpolicies
singular: networkpolicy
preserveUnknownFields: false
scope: Namespaced
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
properties:
egress:
description: The ordered set of egress rules. Each rule contains
a set of packet match criteria and a corresponding action to apply.
items:
description: "A Rule encapsulates a set of match criteria and an
action. Both selector-based security Policy and security Profiles
reference rules - separated out as a list of rules for both ingress
and egress packet matching. \n Each positive match criteria has
a negated version, prefixed with \"Not\". All the match criteria
within a rule must be satisfied for a packet to match. A single
rule can contain the positive and negative version of a match
and both must be satisfied for the rule to match."
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: "NamespaceSelector is an optional field that
contains a selector expression. Only traffic that originates
from (or terminates at) endpoints within the selected
namespaces will be matched. When both NamespaceSelector
and another selector are defined on the same rule, then
only workload endpoints that are matched by both selectors
will be selected by the rule. \n For NetworkPolicy, an
empty NamespaceSelector implies that the Selector is limited
to selecting only workload endpoints in the same namespace
as the NetworkPolicy. \n For NetworkPolicy, `global()`
NamespaceSelector implies that the Selector is limited
to selecting only GlobalNetworkSet or HostEndpoint. \n
For GlobalNetworkPolicy, an empty NamespaceSelector implies
the Selector applies to workload endpoints across all
namespaces."
type: string
nets:
description: Nets is an optional field that restricts the
rule to only apply to traffic that originates from (or
terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: NotPorts is the negated version of the Ports
field. Since only some protocols have ports, if any ports
are specified it requires the Protocol match in the Rule
to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: NotSelector is the negated version of the Selector
field. See Selector field for subtleties with negated
selectors.
type: string
ports:
description: "Ports is an optional field that restricts
the rule to only apply to traffic that has a source (destination)
port that matches one of these ranges/values. This value
is a list of integers or strings that represent ranges
of ports. \n Since only some protocols have ports, if
any ports are specified it requires the Protocol match
in the Rule to be set to \"TCP\" or \"UDP\"."
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for sample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching the selector will be matched. \n Note that: in
addition to the negated version of the Selector (see NotSelector
below), the selector expression syntax itself supports
negation. The two types of negation are subtly different.
One negates the set of matched endpoints, the other negates
the whole match: \n \tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled \tendpoints
that do not have the label \"my_label\". \n \tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled \tendpoints that do have the label \"my_label\".
\n The effect is that the latter will accept packets from
non-Calico sources whereas the former is limited to packets
from Calico-controlled endpoints."
type: string
serviceAccounts:
description: ServiceAccounts is an optional field that restricts
the rule to only apply to traffic that originates from
(or terminates at) a pod running as a matching service
account.
properties:
names:
description: Names is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account whose name is in the list.
items:
type: string
type: array
selector:
description: Selector is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account that matches the given label selector. If
both Names and Selector are specified then they are
AND'ed.
type: string
type: object
services:
description: "Services is an optional field that contains
options for matching Kubernetes Services. If specified,
only traffic that originates from or terminates at endpoints
within the selected service(s) will be matched, and only
to/from each endpoint's port. \n Services cannot be specified
on the same rule as Selector, NotSelector, NamespaceSelector,
Nets, NotNets or ServiceAccounts. \n Ports and NotPorts
can only be specified with Services on ingress rules."
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: Namespace specifies the namespace of the
given Service. If left empty, the rule will match
within this policy's namespace.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: Methods is an optional field that restricts
the rule to apply only to HTTP requests that use one of
the listed HTTP Methods (e.g. GET, PUT, etc.) Multiple
methods are OR'd together.
items:
type: string
type: array
paths:
description: 'Paths is an optional field that restricts
the rule to apply to HTTP requests that use one of the
listed HTTP Paths. Multiple paths are OR''d together.
e.g: - exact: /foo - prefix: /bar NOTE: Each entry may
ONLY specify either a `exact` or a `prefix` match. The
validator will check for it.'
items:
description: 'HTTPPath specifies an HTTP path to match.
It may be either of the form: exact: : which matches
the path exactly or prefix: : which matches
the path prefix'
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
description: ICMP is an optional field that restricts the rule
to apply to a specific type and code of ICMP traffic. This
should only be specified if the Protocol field is set to "ICMP"
or "ICMPv6".
properties:
code:
description: Match on a specific ICMP code. If specified,
the Type value must also be specified. This is a technical
limitation imposed by the kernel's iptables firewall,
which Calico uses to enforce the rule.
type: integer
type:
description: Match on a specific ICMP type. For example
a value of 8 refers to ICMP Echo Request (i.e. pings).
type: integer
type: object
ipVersion:
description: IPVersion is an optional field that restricts the
rule to only match a specific IP version.
type: integer
metadata:
description: Metadata contains additional information for this
rule
properties:
annotations:
additionalProperties:
type: string
description: Annotations is a set of key value pairs that
give extra information about the rule
type: object
type: object
notICMP:
description: NotICMP is the negated version of the ICMP field.
properties:
code:
description: Match on a specific ICMP code. If specified,
the Type value must also be specified. This is a technical
limitation imposed by the kernel's iptables firewall,
which Calico uses to enforce the rule.
type: integer
type:
description: Match on a specific ICMP type. For example
a value of 8 refers to ICMP Echo Request (i.e. pings).
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
description: NotProtocol is the negated version of the Protocol
field.
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
description: "Protocol is an optional field that restricts the
rule to only apply to traffic of a specific IP protocol. Required
if any of the EntityRules contain Ports (because ports only
apply to certain protocols). \n Must be one of these string
values: \"TCP\", \"UDP\", \"ICMP\", \"ICMPv6\", \"SCTP\",
\"UDPLite\" or an integer in the range 1-255."
pattern: ^.*
x-kubernetes-int-or-string: true
source:
description: Source contains the match criteria that apply to
source entity.
properties:
namespaceSelector:
description: "NamespaceSelector is an optional field that
contains a selector expression. Only traffic that originates
from (or terminates at) endpoints within the selected
namespaces will be matched. When both NamespaceSelector
and another selector are defined on the same rule, then
only workload endpoints that are matched by both selectors
will be selected by the rule. \n For NetworkPolicy, an
empty NamespaceSelector implies that the Selector is limited
to selecting only workload endpoints in the same namespace
as the NetworkPolicy. \n For NetworkPolicy, `global()`
NamespaceSelector implies that the Selector is limited
to selecting only GlobalNetworkSet or HostEndpoint. \n
For GlobalNetworkPolicy, an empty NamespaceSelector implies
the Selector applies to workload endpoints across all
namespaces."
type: string
nets:
description: Nets is an optional field that restricts the
rule to only apply to traffic that originates from (or
terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: NotPorts is the negated version of the Ports
field. Since only some protocols have ports, if any ports
are specified it requires the Protocol match in the Rule
to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: NotSelector is the negated version of the Selector
field. See Selector field for subtleties with negated
selectors.
type: string
ports:
description: "Ports is an optional field that restricts
the rule to only apply to traffic that has a source (destination)
port that matches one of these ranges/values. This value
is a list of integers or strings that represent ranges
of ports. \n Since only some protocols have ports, if
any ports are specified it requires the Protocol match
in the Rule to be set to \"TCP\" or \"UDP\"."
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for sample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching the selector will be matched. \n Note that: in
addition to the negated version of the Selector (see NotSelector
below), the selector expression syntax itself supports
negation. The two types of negation are subtly different.
One negates the set of matched endpoints, the other negates
the whole match: \n \tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled \tendpoints
that do not have the label \"my_label\". \n \tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled \tendpoints that do have the label \"my_label\".
\n The effect is that the latter will accept packets from
non-Calico sources whereas the former is limited to packets
from Calico-controlled endpoints."
type: string
serviceAccounts:
description: ServiceAccounts is an optional field that restricts
the rule to only apply to traffic that originates from
(or terminates at) a pod running as a matching service
account.
properties:
names:
description: Names is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account whose name is in the list.
items:
type: string
type: array
selector:
description: Selector is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account that matches the given label selector. If
both Names and Selector are specified then they are
AND'ed.
type: string
type: object
services:
description: "Services is an optional field that contains
options for matching Kubernetes Services. If specified,
only traffic that originates from or terminates at endpoints
within the selected service(s) will be matched, and only
to/from each endpoint's port. \n Services cannot be specified
on the same rule as Selector, NotSelector, NamespaceSelector,
Nets, NotNets or ServiceAccounts. \n Ports and NotPorts
can only be specified with Services on ingress rules."
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: Namespace specifies the namespace of the
given Service. If left empty, the rule will match
within this policy's namespace.
type: string
type: object
type: object
required:
- action
type: object
type: array
ingress:
description: The ordered set of ingress rules. Each rule contains
a set of packet match criteria and a corresponding action to apply.
items:
description: "A Rule encapsulates a set of match criteria and an
action. Both selector-based security Policy and security Profiles
reference rules - separated out as a list of rules for both ingress
and egress packet matching. \n Each positive match criteria has
a negated version, prefixed with \"Not\". All the match criteria
within a rule must be satisfied for a packet to match. A single
rule can contain the positive and negative version of a match
and both must be satisfied for the rule to match."
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: "NamespaceSelector is an optional field that
contains a selector expression. Only traffic that originates
from (or terminates at) endpoints within the selected
namespaces will be matched. When both NamespaceSelector
and another selector are defined on the same rule, then
only workload endpoints that are matched by both selectors
will be selected by the rule. \n For NetworkPolicy, an
empty NamespaceSelector implies that the Selector is limited
to selecting only workload endpoints in the same namespace
as the NetworkPolicy. \n For NetworkPolicy, `global()`
NamespaceSelector implies that the Selector is limited
to selecting only GlobalNetworkSet or HostEndpoint. \n
For GlobalNetworkPolicy, an empty NamespaceSelector implies
the Selector applies to workload endpoints across all
namespaces."
type: string
nets:
description: Nets is an optional field that restricts the
rule to only apply to traffic that originates from (or
terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: NotPorts is the negated version of the Ports
field. Since only some protocols have ports, if any ports
are specified it requires the Protocol match in the Rule
to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: NotSelector is the negated version of the Selector
field. See Selector field for subtleties with negated
selectors.
type: string
ports:
description: "Ports is an optional field that restricts
the rule to only apply to traffic that has a source (destination)
port that matches one of these ranges/values. This value
is a list of integers or strings that represent ranges
of ports. \n Since only some protocols have ports, if
any ports are specified it requires the Protocol match
in the Rule to be set to \"TCP\" or \"UDP\"."
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for sample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching the selector will be matched. \n Note that: in
addition to the negated version of the Selector (see NotSelector
below), the selector expression syntax itself supports
negation. The two types of negation are subtly different.
One negates the set of matched endpoints, the other negates
the whole match: \n \tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled \tendpoints
that do not have the label \"my_label\". \n \tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled \tendpoints that do have the label \"my_label\".
\n The effect is that the latter will accept packets from
non-Calico sources whereas the former is limited to packets
from Calico-controlled endpoints."
type: string
serviceAccounts:
description: ServiceAccounts is an optional field that restricts
the rule to only apply to traffic that originates from
(or terminates at) a pod running as a matching service
account.
properties:
names:
description: Names is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account whose name is in the list.
items:
type: string
type: array
selector:
description: Selector is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account that matches the given label selector. If
both Names and Selector are specified then they are
AND'ed.
type: string
type: object
services:
description: "Services is an optional field that contains
options for matching Kubernetes Services. If specified,
only traffic that originates from or terminates at endpoints
within the selected service(s) will be matched, and only
to/from each endpoint's port. \n Services cannot be specified
on the same rule as Selector, NotSelector, NamespaceSelector,
Nets, NotNets or ServiceAccounts. \n Ports and NotPorts
can only be specified with Services on ingress rules."
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: Namespace specifies the namespace of the
given Service. If left empty, the rule will match
within this policy's namespace.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: Methods is an optional field that restricts
the rule to apply only to HTTP requests that use one of
the listed HTTP Methods (e.g. GET, PUT, etc.) Multiple
methods are OR'd together.
items:
type: string
type: array
paths:
description: 'Paths is an optional field that restricts
the rule to apply to HTTP requests that use one of the
listed HTTP Paths. Multiple paths are OR''d together.
e.g: - exact: /foo - prefix: /bar NOTE: Each entry may
ONLY specify either a `exact` or a `prefix` match. The
validator will check for it.'
items:
description: 'HTTPPath specifies an HTTP path to match.
It may be either of the form: exact: : which matches
the path exactly or prefix: : which matches
the path prefix'
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
description: ICMP is an optional field that restricts the rule
to apply to a specific type and code of ICMP traffic. This
should only be specified if the Protocol field is set to "ICMP"
or "ICMPv6".
properties:
code:
description: Match on a specific ICMP code. If specified,
the Type value must also be specified. This is a technical
limitation imposed by the kernel's iptables firewall,
which Calico uses to enforce the rule.
type: integer
type:
description: Match on a specific ICMP type. For example
a value of 8 refers to ICMP Echo Request (i.e. pings).
type: integer
type: object
ipVersion:
description: IPVersion is an optional field that restricts the
rule to only match a specific IP version.
type: integer
metadata:
description: Metadata contains additional information for this
rule
properties:
annotations:
additionalProperties:
type: string
description: Annotations is a set of key value pairs that
give extra information about the rule
type: object
type: object
notICMP:
description: NotICMP is the negated version of the ICMP field.
properties:
code:
description: Match on a specific ICMP code. If specified,
the Type value must also be specified. This is a technical
limitation imposed by the kernel's iptables firewall,
which Calico uses to enforce the rule.
type: integer
type:
description: Match on a specific ICMP type. For example
a value of 8 refers to ICMP Echo Request (i.e. pings).
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
description: NotProtocol is the negated version of the Protocol
field.
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
description: "Protocol is an optional field that restricts the
rule to only apply to traffic of a specific IP protocol. Required
if any of the EntityRules contain Ports (because ports only
apply to certain protocols). \n Must be one of these string
values: \"TCP\", \"UDP\", \"ICMP\", \"ICMPv6\", \"SCTP\",
\"UDPLite\" or an integer in the range 1-255."
pattern: ^.*
x-kubernetes-int-or-string: true
source:
description: Source contains the match criteria that apply to
source entity.
properties:
namespaceSelector:
description: "NamespaceSelector is an optional field that
contains a selector expression. Only traffic that originates
from (or terminates at) endpoints within the selected
namespaces will be matched. When both NamespaceSelector
and another selector are defined on the same rule, then
only workload endpoints that are matched by both selectors
will be selected by the rule. \n For NetworkPolicy, an
empty NamespaceSelector implies that the Selector is limited
to selecting only workload endpoints in the same namespace
as the NetworkPolicy. \n For NetworkPolicy, `global()`
NamespaceSelector implies that the Selector is limited
to selecting only GlobalNetworkSet or HostEndpoint. \n
For GlobalNetworkPolicy, an empty NamespaceSelector implies
the Selector applies to workload endpoints across all
namespaces."
type: string
nets:
description: Nets is an optional field that restricts the
rule to only apply to traffic that originates from (or
terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: NotPorts is the negated version of the Ports
field. Since only some protocols have ports, if any ports
are specified it requires the Protocol match in the Rule
to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: NotSelector is the negated version of the Selector
field. See Selector field for subtleties with negated
selectors.
type: string
ports:
description: "Ports is an optional field that restricts
the rule to only apply to traffic that has a source (destination)
port that matches one of these ranges/values. This value
is a list of integers or strings that represent ranges
of ports. \n Since only some protocols have ports, if
any ports are specified it requires the Protocol match
in the Rule to be set to \"TCP\" or \"UDP\"."
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for sample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching the selector will be matched. \n Note that: in
addition to the negated version of the Selector (see NotSelector
below), the selector expression syntax itself supports
negation. The two types of negation are subtly different.
One negates the set of matched endpoints, the other negates
the whole match: \n \tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled \tendpoints
that do not have the label \"my_label\". \n \tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled \tendpoints that do have the label \"my_label\".
\n The effect is that the latter will accept packets from
non-Calico sources whereas the former is limited to packets
from Calico-controlled endpoints."
type: string
serviceAccounts:
description: ServiceAccounts is an optional field that restricts
the rule to only apply to traffic that originates from
(or terminates at) a pod running as a matching service
account.
properties:
names:
description: Names is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account whose name is in the list.
items:
type: string
type: array
selector:
description: Selector is an optional field that restricts
the rule to only apply to traffic that originates
from (or terminates at) a pod running as a service
account that matches the given label selector. If
both Names and Selector are specified then they are
AND'ed.
type: string
type: object
services:
description: "Services is an optional field that contains
options for matching Kubernetes Services. If specified,
only traffic that originates from or terminates at endpoints
within the selected service(s) will be matched, and only
to/from each endpoint's port. \n Services cannot be specified
on the same rule as Selector, NotSelector, NamespaceSelector,
Nets, NotNets or ServiceAccounts. \n Ports and NotPorts
can only be specified with Services on ingress rules."
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: Namespace specifies the namespace of the
given Service. If left empty, the rule will match
within this policy's namespace.
type: string
type: object
type: object
required:
- action
type: object
type: array
order:
description: Order is an optional field that specifies the order in
which the policy is applied. Policies with higher "order" are applied
after those with lower order. If the order is omitted, it may be
considered to be "infinite" - i.e. the policy will be applied last. Policies
with identical order will be applied in alphanumerical order based
on the Policy "Name".
type: number
selector:
description: "The selector is an expression used to pick pick out
the endpoints that the policy should be applied to. \n Selector
expressions follow this syntax: \n \tlabel == \"string_literal\"
\ -> comparison, e.g. my_label == \"foo bar\" \tlabel != \"string_literal\"
\ -> not equal; also matches if label is not present \tlabel in
{ \"a\", \"b\", \"c\", ... } -> true if the value of label X is
one of \"a\", \"b\", \"c\" \tlabel not in { \"a\", \"b\", \"c\",
... } -> true if the value of label X is not one of \"a\", \"b\",
\"c\" \thas(label_name) -> True if that label is present \t! expr
-> negation of expr \texpr && expr -> Short-circuit and \texpr
|| expr -> Short-circuit or \t( expr ) -> parens for grouping \tall()
or the empty selector -> matches all endpoints. \n Label names are
allowed to contain alphanumerics, -, _ and /. String literals are
more permissive but they do not support escape characters. \n Examples
(with made-up labels): \n \ttype == \"webserver\" && deployment
== \"prod\" \ttype in {\"frontend\", \"backend\"} \tdeployment !=
\"dev\" \t! has(label_name)"
type: string
serviceAccountSelector:
description: ServiceAccountSelector is an optional field for an expression
used to select a pod based on service accounts.
type: string
types:
description: "Types indicates whether this policy applies to ingress,
or to egress, or to both. When not explicitly specified (and so
the value on creation is empty or nil), Calico defaults Types according
to what Ingress and Egress are present in the policy. The default
is: \n - [ PolicyTypeIngress ], if there are no Egress rules (including
the case where there are also no Ingress rules) \n - [ PolicyTypeEgress
], if there are Egress rules but no Ingress rules \n - [ PolicyTypeIngress,
PolicyTypeEgress ], if there are both Ingress and Egress rules.
\n When the policy is read back again, Types will always be one
of these values, never empty or nil."
items:
description: PolicyType enumerates the possible values of the PolicySpec
Types field.
type: string
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: networksets.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: NetworkSet
listKind: NetworkSetList
plural: networksets
singular: networkset
preserveUnknownFields: false
scope: Namespaced
versions:
- name: v1
schema:
openAPIV3Schema:
description: NetworkSet is the Namespaced-equivalent of the GlobalNetworkSet.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: NetworkSetSpec contains the specification for a NetworkSet
resource.
properties:
nets:
description: The list of IP networks that belong to this set.
items:
type: string
type: array
type: object
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
# Source: calico/templates/calico-kube-controllers-rbac.yaml
# Include a clusterrole for the kube-controllers component,
# and bind it to the calico-kube-controllers serviceaccount.
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: calico-kube-controllers
rules:
# Nodes are watched to monitor for deletions.
- apiGroups: [""]
resources:
- nodes
verbs:
- watch
- list
- get
# Pods are watched to check for existence as part of IPAM controller.
- apiGroups: [""]
resources:
- pods
verbs:
- get
- list
- watch
# IPAM resources are manipulated in response to node and block updates, as well as periodic triggers.
- apiGroups: ["crd.projectcalico.org"]
resources:
- ipreservations
verbs:
- list
- apiGroups: ["crd.projectcalico.org"]
resources:
- blockaffinities
- ipamblocks
- ipamhandles
verbs:
- get
- list
- create
- update
- delete
- watch
# Pools are watched to maintain a mapping of blocks to IP pools.
- apiGroups: ["crd.projectcalico.org"]
resources:
- ippools
verbs:
- list
- watch
# kube-controllers manages hostendpoints.
- apiGroups: ["crd.projectcalico.org"]
resources:
- hostendpoints
verbs:
- get
- list
- create
- update
- delete
# Needs access to update clusterinformations.
- apiGroups: ["crd.projectcalico.org"]
resources:
- clusterinformations
verbs:
- get
- list
- create
- update
- watch
# KubeControllersConfiguration is where it gets its config
- apiGroups: ["crd.projectcalico.org"]
resources:
- kubecontrollersconfigurations
verbs:
# read its own config
- get
# create a default if none exists
- create
# update status
- update
# watch for changes
- watch
---
# Source: calico/templates/calico-node-rbac.yaml
# Include a clusterrole for the calico-node DaemonSet,
# and bind it to the calico-node serviceaccount.
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: calico-node
rules:
# Used for creating service account tokens to be used by the CNI plugin
- apiGroups: [""]
resources:
- serviceaccounts/token
resourceNames:
- calico-node
verbs:
- create
# The CNI plugin needs to get pods, nodes, and namespaces.
- apiGroups: [""]
resources:
- pods
- nodes
- namespaces
verbs:
- get
# EndpointSlices are used for Service-based network policy rule
# enforcement.
- apiGroups: ["discovery.k8s.io"]
resources:
- endpointslices
verbs:
- watch
- list
- apiGroups: [""]
resources:
- endpoints
- services
verbs:
# Used to discover service IPs for advertisement.
- watch
- list
# Used to discover Typhas.
- get
# Pod CIDR auto-detection on kubeadm needs access to config maps.
- apiGroups: [""]
resources:
- configmaps
verbs:
- get
- apiGroups: [""]
resources:
- nodes/status
verbs:
# Needed for clearing NodeNetworkUnavailable flag.
- patch
# Calico stores some configuration information in node annotations.
- update
# Watch for changes to Kubernetes NetworkPolicies.
- apiGroups: ["networking.k8s.io"]
resources:
- networkpolicies
verbs:
- watch
- list
# Used by Calico for policy information.
- apiGroups: [""]
resources:
- pods
- namespaces
- serviceaccounts
verbs:
- list
- watch
# The CNI plugin patches pods/status.
- apiGroups: [""]
resources:
- pods/status
verbs:
- patch
# Calico monitors various CRDs for config.
- apiGroups: ["crd.projectcalico.org"]
resources:
- globalfelixconfigs
- felixconfigurations
- bgppeers
- globalbgpconfigs
- bgpconfigurations
- ippools
- ipreservations
- ipamblocks
- globalnetworkpolicies
- globalnetworksets
- networkpolicies
- networksets
- clusterinformations
- hostendpoints
- blockaffinities
- caliconodestatuses
verbs:
- get
- list
- watch
# Calico must create and update some CRDs on startup.
- apiGroups: ["crd.projectcalico.org"]
resources:
- ippools
- felixconfigurations
- clusterinformations
verbs:
- create
- update
# Calico must update some CRDs.
- apiGroups: [ "crd.projectcalico.org" ]
resources:
- caliconodestatuses
verbs:
- update
# Calico stores some configuration information on the node.
- apiGroups: [""]
resources:
- nodes
verbs:
- get
- list
- watch
# These permissions are only required for upgrade from v2.6, and can
# be removed after upgrade or on fresh installations.
- apiGroups: ["crd.projectcalico.org"]
resources:
- bgpconfigurations
- bgppeers
verbs:
- create
- update
# These permissions are required for Calico CNI to perform IPAM allocations.
- apiGroups: ["crd.projectcalico.org"]
resources:
- blockaffinities
- ipamblocks
- ipamhandles
verbs:
- get
- list
- create
- update
- delete
# The CNI plugin and calico/node need to be able to create a default
# IPAMConfiguration
- apiGroups: ["crd.projectcalico.org"]
resources:
- ipamconfigs
verbs:
- get
- create
# Block affinities must also be watchable by confd for route aggregation.
- apiGroups: ["crd.projectcalico.org"]
resources:
- blockaffinities
verbs:
- watch
# The Calico IPAM migration needs to get daemonsets. These permissions can be
# removed if not upgrading from an installation using host-local IPAM.
- apiGroups: ["apps"]
resources:
- daemonsets
verbs:
- get
---
# Source: calico/templates/calico-kube-controllers-rbac.yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: calico-kube-controllers
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: calico-kube-controllers
subjects:
- kind: ServiceAccount
name: calico-kube-controllers
namespace: kube-system
---
# Source: calico/templates/calico-node-rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: calico-node
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: calico-node
subjects:
- kind: ServiceAccount
name: calico-node
namespace: kube-system
---
# Source: calico/templates/calico-node.yaml
# This manifest installs the calico-node container, as well
# as the CNI plugins and network config on
# each master and worker node in a Kubernetes cluster.
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: calico-node
namespace: kube-system
labels:
k8s-app: calico-node
spec:
selector:
matchLabels:
k8s-app: calico-node
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
template:
metadata:
labels:
k8s-app: calico-node
spec:
nodeSelector:
kubernetes.io/os: linux
hostNetwork: true
tolerations:
# Make sure calico-node gets scheduled on all nodes.
- effect: NoSchedule
operator: Exists
# Mark the pod as a critical add-on for rescheduling.
- key: CriticalAddonsOnly
operator: Exists
- effect: NoExecute
operator: Exists
serviceAccountName: calico-node
# Minimize downtime during a rolling upgrade or deletion; tell Kubernetes to do a "force
# deletion": https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods.
terminationGracePeriodSeconds: 0
priorityClassName: system-node-critical
initContainers:
# This container performs upgrade from host-local IPAM to calico-ipam.
# It can be deleted if this is a fresh installation, or if you have already
# upgraded to use calico-ipam.
- name: upgrade-ipam
image: docker.io/calico/cni:v3.25.0
imagePullPolicy: IfNotPresent
command: ["/opt/cni/bin/calico-ipam", "-upgrade"]
envFrom:
- configMapRef:
# Allow KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT to be overridden for eBPF mode.
name: kubernetes-services-endpoint
optional: true
env:
- name: KUBERNETES_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: CALICO_NETWORKING_BACKEND
valueFrom:
configMapKeyRef:
name: calico-config
key: calico_backend
volumeMounts:
- mountPath: /var/lib/cni/networks
name: host-local-net-dir
- mountPath: /host/opt/cni/bin
name: cni-bin-dir
securityContext:
privileged: true
# This container installs the CNI binaries
# and CNI network config file on each node.
- name: install-cni
image: docker.io/calico/cni:v3.25.0
imagePullPolicy: IfNotPresent
command: ["/opt/cni/bin/install"]
envFrom:
- configMapRef:
# Allow KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT to be overridden for eBPF mode.
name: kubernetes-services-endpoint
optional: true
env:
# Name of the CNI config file to create.
- name: CNI_CONF_NAME
value: "10-calico.conflist"
# The CNI network config to install on each node.
- name: CNI_NETWORK_CONFIG
valueFrom:
configMapKeyRef:
name: calico-config
key: cni_network_config
# Set the hostname based on the k8s node name.
- name: KUBERNETES_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
# CNI MTU Config variable
- name: CNI_MTU
valueFrom:
configMapKeyRef:
name: calico-config
key: veth_mtu
# Prevents the container from sleeping forever.
- name: SLEEP
value: "false"
volumeMounts:
- mountPath: /host/opt/cni/bin
name: cni-bin-dir
- mountPath: /host/etc/cni/net.d
name: cni-net-dir
securityContext:
privileged: true
# This init container mounts the necessary filesystems needed by the BPF data plane
# i.e. bpf at /sys/fs/bpf and cgroup2 at /run/calico/cgroup. Calico-node initialisation is executed
# in best effort fashion, i.e. no failure for errors, to not disrupt pod creation in iptable mode.
- name: "mount-bpffs"
image: docker.io/calico/node:v3.25.0
imagePullPolicy: IfNotPresent
command: ["calico-node", "-init", "-best-effort"]
volumeMounts:
- mountPath: /sys/fs
name: sys-fs
# Bidirectional is required to ensure that the new mount we make at /sys/fs/bpf propagates to the host
# so that it outlives the init container.
mountPropagation: Bidirectional
- mountPath: /var/run/calico
name: var-run-calico
# Bidirectional is required to ensure that the new mount we make at /run/calico/cgroup propagates to the host
# so that it outlives the init container.
mountPropagation: Bidirectional
# Mount /proc/ from host which usually is an init program at /nodeproc. It's needed by mountns binary,
# executed by calico-node, to mount root cgroup2 fs at /run/calico/cgroup to attach CTLB programs correctly.
- mountPath: /nodeproc
name: nodeproc
readOnly: true
securityContext:
privileged: true
containers:
# Runs calico-node container on each Kubernetes node. This
# container programs network policy and routes on each
# host.
- name: calico-node
image: docker.io/calico/node:v3.25.0
imagePullPolicy: IfNotPresent
envFrom:
- configMapRef:
# Allow KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT to be overridden for eBPF mode.
name: kubernetes-services-endpoint
optional: true
env:
# Use Kubernetes API as the backing datastore.
- name: DATASTORE_TYPE
value: "kubernetes"
# Wait for the datastore.
- name: WAIT_FOR_DATASTORE
value: "true"
# Set based on the k8s node name.
- name: NODENAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
# Choose the backend to use.
- name: CALICO_NETWORKING_BACKEND
valueFrom:
configMapKeyRef:
name: calico-config
key: calico_backend
# Cluster type to identify the deployment type
- name: CLUSTER_TYPE
value: "k8s,bgp"
# Auto-detect the BGP IP address.
- name: IP
value: "autodetect"
# Enable IPIP
- name: CALICO_IPV4POOL_IPIP
value: "Always"
# Enable or Disable VXLAN on the default IP pool.
- name: CALICO_IPV4POOL_VXLAN
value: "Never"
# Enable or Disable VXLAN on the default IPv6 IP pool.
- name: CALICO_IPV6POOL_VXLAN
value: "Never"
# Set MTU for tunnel device used if ipip is enabled
- name: FELIX_IPINIPMTU
valueFrom:
configMapKeyRef:
name: calico-config
key: veth_mtu
# Set MTU for the VXLAN tunnel device.
- name: FELIX_VXLANMTU
valueFrom:
configMapKeyRef:
name: calico-config
key: veth_mtu
# Set MTU for the Wireguard tunnel device.
- name: FELIX_WIREGUARDMTU
valueFrom:
configMapKeyRef:
name: calico-config
key: veth_mtu
# The default IPv4 pool to create on startup if none exists. Pod IPs will be
# chosen from this range. Changing this value after installation will have
# no effect. This should fall within `--cluster-cidr`.
- name: CALICO_IPV4POOL_CIDR
value: "10.244.0.0/16"
# Disable file logging so `kubectl logs` works.
- name: CALICO_DISABLE_FILE_LOGGING
value: "true"
# Set Felix endpoint to host default action to ACCEPT.
- name: FELIX_DEFAULTENDPOINTTOHOSTACTION
value: "ACCEPT"
# Disable IPv6 on Kubernetes.
- name: FELIX_IPV6SUPPORT
value: "false"
- name: FELIX_HEALTHENABLED
value: "true"
securityContext:
privileged: true
resources:
requests:
cpu: 250m
lifecycle:
preStop:
exec:
command:
- /bin/calico-node
- -shutdown
livenessProbe:
exec:
command:
- /bin/calico-node
- -felix-live
- -bird-live
periodSeconds: 10
initialDelaySeconds: 10
failureThreshold: 6
timeoutSeconds: 10
readinessProbe:
exec:
command:
- /bin/calico-node
- -felix-ready
- -bird-ready
periodSeconds: 10
timeoutSeconds: 10
volumeMounts:
# For maintaining CNI plugin API credentials.
- mountPath: /host/etc/cni/net.d
name: cni-net-dir
readOnly: false
- mountPath: /lib/modules
name: lib-modules
readOnly: true
- mountPath: /run/xtables.lock
name: xtables-lock
readOnly: false
- mountPath: /var/run/calico
name: var-run-calico
readOnly: false
- mountPath: /var/lib/calico
name: var-lib-calico
readOnly: false
- name: policysync
mountPath: /var/run/nodeagent
# For eBPF mode, we need to be able to mount the BPF filesystem at /sys/fs/bpf so we mount in the
# parent directory.
- name: bpffs
mountPath: /sys/fs/bpf
- name: cni-log-dir
mountPath: /var/log/calico/cni
readOnly: true
volumes:
# Used by calico-node.
- name: lib-modules
hostPath:
path: /lib/modules
- name: var-run-calico
hostPath:
path: /var/run/calico
- name: var-lib-calico
hostPath:
path: /var/lib/calico
- name: xtables-lock
hostPath:
path: /run/xtables.lock
type: FileOrCreate
- name: sys-fs
hostPath:
path: /sys/fs/
type: DirectoryOrCreate
- name: bpffs
hostPath:
path: /sys/fs/bpf
type: Directory
# mount /proc at /nodeproc to be used by mount-bpffs initContainer to mount root cgroup2 fs.
- name: nodeproc
hostPath:
path: /proc
# Used to install CNI.
- name: cni-bin-dir
hostPath:
path: /opt/cni/bin
- name: cni-net-dir
hostPath:
path: /etc/cni/net.d
# Used to access CNI logs.
- name: cni-log-dir
hostPath:
path: /var/log/calico/cni
# Mount in the directory for host-local IPAM allocations. This is
# used when upgrading from host-local to calico-ipam, and can be removed
# if not using the upgrade-ipam init container.
- name: host-local-net-dir
hostPath:
path: /var/lib/cni/networks
# Used to create per-pod Unix Domain Sockets
- name: policysync
hostPath:
type: DirectoryOrCreate
path: /var/run/nodeagent
---
# Source: calico/templates/calico-kube-controllers.yaml
# See https://github.com/projectcalico/kube-controllers
apiVersion: apps/v1
kind: Deployment
metadata:
name: calico-kube-controllers
namespace: kube-system
labels:
k8s-app: calico-kube-controllers
spec:
# The controllers can only have a single active instance.
replicas: 1
selector:
matchLabels:
k8s-app: calico-kube-controllers
strategy:
type: Recreate
template:
metadata:
name: calico-kube-controllers
namespace: kube-system
labels:
k8s-app: calico-kube-controllers
spec:
nodeSelector:
kubernetes.io/os: linux
tolerations:
# Mark the pod as a critical add-on for rescheduling.
- key: CriticalAddonsOnly
operator: Exists
- key: node-role.kubernetes.io/master
effect: NoSchedule
- key: node-role.kubernetes.io/control-plane
effect: NoSchedule
serviceAccountName: calico-kube-controllers
priorityClassName: system-cluster-critical
containers:
- name: calico-kube-controllers
image: docker.io/calico/kube-controllers:v3.25.0
imagePullPolicy: IfNotPresent
env:
# Choose which controllers to run.
- name: ENABLED_CONTROLLERS
value: node
- name: DATASTORE_TYPE
value: kubernetes
livenessProbe:
exec:
command:
- /usr/bin/check-status
- -l
periodSeconds: 10
initialDelaySeconds: 10
failureThreshold: 6
timeoutSeconds: 10
readinessProbe:
exec:
command:
- /usr/bin/check-status
- -r
periodSeconds: 10
================================================
FILE: resources/custom-cni/canal.yaml
================================================
---
# Source: calico/templates/calico-kube-controllers.yaml
# This manifest creates a Pod Disruption Budget for Controller to allow K8s Cluster Autoscaler to evict
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: calico-kube-controllers
namespace: kube-system
labels:
k8s-app: calico-kube-controllers
spec:
maxUnavailable: 1
selector:
matchLabels:
k8s-app: calico-kube-controllers
---
# Source: calico/templates/calico-kube-controllers.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: calico-kube-controllers
namespace: kube-system
---
# Source: calico/templates/calico-node.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: canal
namespace: kube-system
---
# Source: calico/templates/calico-node.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: calico-cni-plugin
namespace: kube-system
---
# Source: calico/templates/calico-config.yaml
# This ConfigMap is used to configure a self-hosted Canal installation.
kind: ConfigMap
apiVersion: v1
metadata:
name: canal-config
namespace: kube-system
data:
# Typha is disabled.
typha_service_name: "none"
# The interface used by canal for host <-> host communication.
# If left blank, then the interface is chosen using the node's
# default route.
canal_iface: ""
# Whether or not to masquerade traffic to destinations not within
# the pod network.
masquerade: "true"
# Configure the MTU to use for workload interfaces and tunnels.
# By default, MTU is auto-detected, and explicitly setting this field should not be required.
# You can override auto-detection by providing a non-zero value.
veth_mtu: "0"
# The CNI network configuration to install on each node. The special
# values in this config will be automatically populated.
cni_network_config: |-
{
"name": "k8s-pod-network",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "calico",
"log_level": "info",
"log_file_path": "/var/log/calico/cni/cni.log",
"datastore_type": "kubernetes",
"nodename": "__KUBERNETES_NODE_NAME__",
"mtu": __CNI_MTU__,
"ipam": {
"type": "host-local",
"subnet": "usePodCidr"
},
"policy": {
"type": "k8s"
},
"kubernetes": {
"kubeconfig": "__KUBECONFIG_FILEPATH__"
}
},
{
"type": "portmap",
"snat": true,
"capabilities": {"portMappings": true}
}
]
}
# Flannel network configuration. Mounted into the flannel container.
net-conf.json: |
{
"Network": "172.17.0.0/16",
"Backend": {
"Type": "vxlan"
}
}
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: bgpconfigurations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BGPConfiguration
listKind: BGPConfigurationList
plural: bgpconfigurations
singular: bgpconfiguration
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
description: BGPConfiguration contains the configuration for any BGP routing.
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: BGPConfigurationSpec contains the values of the BGP configuration.
properties:
asNumber:
description: 'ASNumber is the default AS number used by a node. [Default:
64512]'
format: int32
type: integer
bindMode:
description: |-
BindMode indicates whether to listen for BGP connections on all addresses (None)
or only on the node's canonical IP address Node.Spec.BGP.IPvXAddress (NodeIP).
Default behaviour is to listen for BGP connections on all addresses.
type: string
communities:
description: Communities is a list of BGP community values and their
arbitrary names for tagging routes.
items:
description: Community contains standard or large community value
and its name.
properties:
name:
description: Name given to community value.
type: string
value:
description: |-
Value must be of format `aa:nn` or `aa:nn:mm`.
For standard community use `aa:nn` format, where `aa` and `nn` are 16 bit number.
For large community use `aa:nn:mm` format, where `aa`, `nn` and `mm` are 32 bit number.
Where, `aa` is an AS Number, `nn` and `mm` are per-AS identifier.
pattern: ^(\d+):(\d+)$|^(\d+):(\d+):(\d+)$
type: string
type: object
type: array
ignoredInterfaces:
description: IgnoredInterfaces indicates the network interfaces that
needs to be excluded when reading device routes.
items:
type: string
type: array
listenPort:
description: ListenPort is the port where BGP protocol should listen.
Defaults to 179
maximum: 65535
minimum: 1
type: integer
localWorkloadPeeringIPV4:
description: |-
The virtual IPv4 address of the node with which its local workload is expected to peer.
It is recommended to use a link-local address.
type: string
localWorkloadPeeringIPV6:
description: |-
The virtual IPv6 address of the node with which its local workload is expected to peer.
It is recommended to use a link-local address.
type: string
logSeverityScreen:
description: 'LogSeverityScreen is the log severity above which logs
are sent to the stdout. [Default: INFO]'
type: string
nodeMeshMaxRestartTime:
description: |-
Time to allow for software restart for node-to-mesh peerings. When specified, this is configured
as the graceful restart timeout. When not specified, the BIRD default of 120s is used.
This field can only be set on the default BGPConfiguration instance and requires that NodeMesh is enabled
type: string
nodeMeshPassword:
description: |-
Optional BGP password for full node-to-mesh peerings.
This field can only be set on the default BGPConfiguration instance and requires that NodeMesh is enabled
properties:
secretKeyRef:
description: Selects a key of a secret in the node pod's namespace.
properties:
key:
description: The key of the secret to select from. Must be
a valid secret key.
type: string
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
optional:
description: Specify whether the Secret or its key must be
defined
type: boolean
required:
- key
type: object
x-kubernetes-map-type: atomic
type: object
nodeToNodeMeshEnabled:
description: 'NodeToNodeMeshEnabled sets whether full node to node
BGP mesh is enabled. [Default: true]'
type: boolean
prefixAdvertisements:
description: PrefixAdvertisements contains per-prefix advertisement
configuration.
items:
description: PrefixAdvertisement configures advertisement properties
for the specified CIDR.
properties:
cidr:
description: CIDR for which properties should be advertised.
type: string
communities:
description: |-
Communities can be list of either community names already defined in `Specs.Communities` or community value of format `aa:nn` or `aa:nn:mm`.
For standard community use `aa:nn` format, where `aa` and `nn` are 16 bit number.
For large community use `aa:nn:mm` format, where `aa`, `nn` and `mm` are 32 bit number.
Where,`aa` is an AS Number, `nn` and `mm` are per-AS identifier.
items:
type: string
type: array
type: object
type: array
serviceClusterIPs:
description: |-
ServiceClusterIPs are the CIDR blocks from which service cluster IPs are allocated.
If specified, Calico will advertise these blocks, as well as any cluster IPs within them.
items:
description: ServiceClusterIPBlock represents a single allowed ClusterIP
CIDR block.
properties:
cidr:
type: string
type: object
type: array
serviceExternalIPs:
description: |-
ServiceExternalIPs are the CIDR blocks for Kubernetes Service External IPs.
Kubernetes Service ExternalIPs will only be advertised if they are within one of these blocks.
items:
description: ServiceExternalIPBlock represents a single allowed
External IP CIDR block.
properties:
cidr:
type: string
type: object
type: array
serviceLoadBalancerIPs:
description: |-
ServiceLoadBalancerIPs are the CIDR blocks for Kubernetes Service LoadBalancer IPs.
Kubernetes Service status.LoadBalancer.Ingress IPs will only be advertised if they are within one of these blocks.
items:
description: ServiceLoadBalancerIPBlock represents a single allowed
LoadBalancer IP CIDR block.
properties:
cidr:
type: string
type: object
type: array
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: bgpfilters.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BGPFilter
listKind: BGPFilterList
plural: bgpfilters
singular: bgpfilter
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: BGPFilterSpec contains the IPv4 and IPv6 filter rules of
the BGP Filter.
properties:
exportV4:
description: The ordered set of IPv4 BGPFilter rules acting on exporting
routes to a peer.
items:
description: BGPFilterRuleV4 defines a BGP filter rule consisting
a single IPv4 CIDR block and a filter action for this CIDR.
properties:
action:
type: string
cidr:
type: string
interface:
type: string
matchOperator:
type: string
prefixLength:
properties:
max:
format: int32
maximum: 32
minimum: 0
type: integer
min:
format: int32
maximum: 32
minimum: 0
type: integer
type: object
source:
type: string
required:
- action
type: object
type: array
exportV6:
description: The ordered set of IPv6 BGPFilter rules acting on exporting
routes to a peer.
items:
description: BGPFilterRuleV6 defines a BGP filter rule consisting
a single IPv6 CIDR block and a filter action for this CIDR.
properties:
action:
type: string
cidr:
type: string
interface:
type: string
matchOperator:
type: string
prefixLength:
properties:
max:
format: int32
maximum: 128
minimum: 0
type: integer
min:
format: int32
maximum: 128
minimum: 0
type: integer
type: object
source:
type: string
required:
- action
type: object
type: array
importV4:
description: The ordered set of IPv4 BGPFilter rules acting on importing
routes from a peer.
items:
description: BGPFilterRuleV4 defines a BGP filter rule consisting
a single IPv4 CIDR block and a filter action for this CIDR.
properties:
action:
type: string
cidr:
type: string
interface:
type: string
matchOperator:
type: string
prefixLength:
properties:
max:
format: int32
maximum: 32
minimum: 0
type: integer
min:
format: int32
maximum: 32
minimum: 0
type: integer
type: object
source:
type: string
required:
- action
type: object
type: array
importV6:
description: The ordered set of IPv6 BGPFilter rules acting on importing
routes from a peer.
items:
description: BGPFilterRuleV6 defines a BGP filter rule consisting
a single IPv6 CIDR block and a filter action for this CIDR.
properties:
action:
type: string
cidr:
type: string
interface:
type: string
matchOperator:
type: string
prefixLength:
properties:
max:
format: int32
maximum: 128
minimum: 0
type: integer
min:
format: int32
maximum: 128
minimum: 0
type: integer
type: object
source:
type: string
required:
- action
type: object
type: array
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: bgppeers.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BGPPeer
listKind: BGPPeerList
plural: bgppeers
singular: bgppeer
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: BGPPeerSpec contains the specification for a BGPPeer resource.
properties:
asNumber:
description: The AS Number of the peer.
format: int32
type: integer
filters:
description: The ordered set of BGPFilters applied on this BGP peer.
items:
type: string
type: array
keepOriginalNextHop:
description: |-
Option to keep the original nexthop field when routes are sent to a BGP Peer.
Setting "true" configures the selected BGP Peers node to use the "next hop keep;"
instead of "next hop self;"(default) in the specific branch of the Node on "bird.cfg".
type: boolean
localWorkloadSelector:
description: |-
Selector for the local workload that the node should peer with. When this is set, the peerSelector and peerIP fields must be empty,
and the ASNumber must not be empty.
type: string
maxRestartTime:
description: |-
Time to allow for software restart. When specified, this is configured as the graceful
restart timeout. When not specified, the BIRD default of 120s is used.
type: string
node:
description: |-
The node name identifying the Calico node instance that is targeted by this peer.
If this is not set, and no nodeSelector is specified, then this BGP peer selects all
nodes in the cluster.
type: string
nodeSelector:
description: |-
Selector for the nodes that should have this peering. When this is set, the Node
field must be empty.
type: string
numAllowedLocalASNumbers:
description: |-
Maximum number of local AS numbers that are allowed in the AS path for received routes.
This removes BGP loop prevention and should only be used if absolutely necessary.
format: int32
type: integer
password:
description: Optional BGP password for the peerings generated by this
BGPPeer resource.
properties:
secretKeyRef:
description: Selects a key of a secret in the node pod's namespace.
properties:
key:
description: The key of the secret to select from. Must be
a valid secret key.
type: string
name:
default: ""
description: |-
Name of the referent.
This field is effectively required, but due to backwards compatibility is
allowed to be empty. Instances of this type with an empty value here are
almost certainly wrong.
More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
type: string
optional:
description: Specify whether the Secret or its key must be
defined
type: boolean
required:
- key
type: object
x-kubernetes-map-type: atomic
type: object
peerIP:
description: |-
The IP address of the peer followed by an optional port number to peer with.
If port number is given, format should be `[]:port` or `:` for IPv4.
If optional port number is not set, and this peer IP and ASNumber belongs to a calico/node
with ListenPort set in BGPConfiguration, then we use that port to peer.
type: string
peerSelector:
description: |-
Selector for the remote nodes to peer with. When this is set, the PeerIP and
ASNumber fields must be empty. For each peering between the local node and
selected remote nodes, we configure an IPv4 peering if both ends have
NodeBGPSpec.IPv4Address specified, and an IPv6 peering if both ends have
NodeBGPSpec.IPv6Address specified. The remote AS number comes from the remote
node's NodeBGPSpec.ASNumber, or the global default if that is not set.
type: string
reachableBy:
description: |-
Add an exact, i.e. /32, static route toward peer IP in order to prevent route flapping.
ReachableBy contains the address of the gateway which peer can be reached by.
type: string
sourceAddress:
description: |-
Specifies whether and how to configure a source address for the peerings generated by
this BGPPeer resource. Default value "UseNodeIP" means to configure the node IP as the
source address. "None" means not to configure a source address.
type: string
ttlSecurity:
description: |-
TTLSecurity enables the generalized TTL security mechanism (GTSM) which protects against spoofed packets by
ignoring received packets with a smaller than expected TTL value. The provided value is the number of hops
(edges) between the peers.
type: integer
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: blockaffinities.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: BlockAffinity
listKind: BlockAffinityList
plural: blockaffinities
singular: blockaffinity
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: BlockAffinitySpec contains the specification for a BlockAffinity
resource.
properties:
cidr:
type: string
deleted:
description: |-
Deleted indicates that this block affinity is being deleted.
This field is a string for compatibility with older releases that
mistakenly treat this field as a string.
type: string
node:
type: string
state:
type: string
type:
type: string
required:
- cidr
- deleted
- node
- state
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: caliconodestatuses.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: CalicoNodeStatus
listKind: CalicoNodeStatusList
plural: caliconodestatuses
singular: caliconodestatus
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: CalicoNodeStatusSpec contains the specification for a CalicoNodeStatus
resource.
properties:
classes:
description: |-
Classes declares the types of information to monitor for this calico/node,
and allows for selective status reporting about certain subsets of information.
items:
type: string
type: array
node:
description: The node name identifies the Calico node instance for
node status.
type: string
updatePeriodSeconds:
description: |-
UpdatePeriodSeconds is the period at which CalicoNodeStatus should be updated.
Set to 0 to disable CalicoNodeStatus refresh. Maximum update period is one day.
format: int32
type: integer
type: object
status:
description: |-
CalicoNodeStatusStatus defines the observed state of CalicoNodeStatus.
No validation needed for status since it is updated by Calico.
properties:
agent:
description: Agent holds agent status on the node.
properties:
birdV4:
description: BIRDV4 represents the latest observed status of bird4.
properties:
lastBootTime:
description: LastBootTime holds the value of lastBootTime
from bird.ctl output.
type: string
lastReconfigurationTime:
description: LastReconfigurationTime holds the value of lastReconfigTime
from bird.ctl output.
type: string
routerID:
description: Router ID used by bird.
type: string
state:
description: The state of the BGP Daemon.
type: string
version:
description: Version of the BGP daemon
type: string
type: object
birdV6:
description: BIRDV6 represents the latest observed status of bird6.
properties:
lastBootTime:
description: LastBootTime holds the value of lastBootTime
from bird.ctl output.
type: string
lastReconfigurationTime:
description: LastReconfigurationTime holds the value of lastReconfigTime
from bird.ctl output.
type: string
routerID:
description: Router ID used by bird.
type: string
state:
description: The state of the BGP Daemon.
type: string
version:
description: Version of the BGP daemon
type: string
type: object
type: object
bgp:
description: BGP holds node BGP status.
properties:
numberEstablishedV4:
description: The total number of IPv4 established bgp sessions.
type: integer
numberEstablishedV6:
description: The total number of IPv6 established bgp sessions.
type: integer
numberNotEstablishedV4:
description: The total number of IPv4 non-established bgp sessions.
type: integer
numberNotEstablishedV6:
description: The total number of IPv6 non-established bgp sessions.
type: integer
peersV4:
description: PeersV4 represents IPv4 BGP peers status on the node.
items:
description: CalicoNodePeer contains the status of BGP peers
on the node.
properties:
peerIP:
description: IP address of the peer whose condition we are
reporting.
type: string
since:
description: Since the state or reason last changed.
type: string
state:
description: State is the BGP session state.
type: string
type:
description: |-
Type indicates whether this peer is configured via the node-to-node mesh,
or via en explicit global or per-node BGPPeer object.
type: string
type: object
type: array
peersV6:
description: PeersV6 represents IPv6 BGP peers status on the node.
items:
description: CalicoNodePeer contains the status of BGP peers
on the node.
properties:
peerIP:
description: IP address of the peer whose condition we are
reporting.
type: string
since:
description: Since the state or reason last changed.
type: string
state:
description: State is the BGP session state.
type: string
type:
description: |-
Type indicates whether this peer is configured via the node-to-node mesh,
or via en explicit global or per-node BGPPeer object.
type: string
type: object
type: array
required:
- numberEstablishedV4
- numberEstablishedV6
- numberNotEstablishedV4
- numberNotEstablishedV6
type: object
lastUpdated:
description: |-
LastUpdated is a timestamp representing the server time when CalicoNodeStatus object
last updated. It is represented in RFC3339 form and is in UTC.
format: date-time
nullable: true
type: string
routes:
description: Routes reports routes known to the Calico BGP daemon
on the node.
properties:
routesV4:
description: RoutesV4 represents IPv4 routes on the node.
items:
description: CalicoNodeRoute contains the status of BGP routes
on the node.
properties:
destination:
description: Destination of the route.
type: string
gateway:
description: Gateway for the destination.
type: string
interface:
description: Interface for the destination
type: string
learnedFrom:
description: LearnedFrom contains information regarding
where this route originated.
properties:
peerIP:
description: If sourceType is NodeMesh or BGPPeer, IP
address of the router that sent us this route.
type: string
sourceType:
description: Type of the source where a route is learned
from.
type: string
type: object
type:
description: Type indicates if the route is being used for
forwarding or not.
type: string
type: object
type: array
routesV6:
description: RoutesV6 represents IPv6 routes on the node.
items:
description: CalicoNodeRoute contains the status of BGP routes
on the node.
properties:
destination:
description: Destination of the route.
type: string
gateway:
description: Gateway for the destination.
type: string
interface:
description: Interface for the destination
type: string
learnedFrom:
description: LearnedFrom contains information regarding
where this route originated.
properties:
peerIP:
description: If sourceType is NodeMesh or BGPPeer, IP
address of the router that sent us this route.
type: string
sourceType:
description: Type of the source where a route is learned
from.
type: string
type: object
type:
description: Type indicates if the route is being used for
forwarding or not.
type: string
type: object
type: array
type: object
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: clusterinformations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: ClusterInformation
listKind: ClusterInformationList
plural: clusterinformations
singular: clusterinformation
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
description: ClusterInformation contains the cluster specific information.
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: ClusterInformationSpec contains the values of describing
the cluster.
properties:
calicoVersion:
description: CalicoVersion is the version of Calico that the cluster
is running
type: string
clusterGUID:
description: ClusterGUID is the GUID of the cluster
type: string
clusterType:
description: ClusterType describes the type of the cluster
type: string
datastoreReady:
description: |-
DatastoreReady is used during significant datastore migrations to signal to components
such as Felix that it should wait before accessing the datastore.
type: boolean
variant:
description: Variant declares which variant of Calico should be active.
type: string
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: felixconfigurations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: FelixConfiguration
listKind: FelixConfigurationList
plural: felixconfigurations
singular: felixconfiguration
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
description: Felix Configuration contains the configuration for Felix.
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: FelixConfigurationSpec contains the values of the Felix configuration.
properties:
allowIPIPPacketsFromWorkloads:
description: |-
AllowIPIPPacketsFromWorkloads controls whether Felix will add a rule to drop IPIP encapsulated traffic
from workloads. [Default: false]
type: boolean
allowVXLANPacketsFromWorkloads:
description: |-
AllowVXLANPacketsFromWorkloads controls whether Felix will add a rule to drop VXLAN encapsulated traffic
from workloads. [Default: false]
type: boolean
awsSrcDstCheck:
description: |-
AWSSrcDstCheck controls whether Felix will try to change the "source/dest check" setting on the EC2 instance
on which it is running. A value of "Disable" will try to disable the source/dest check. Disabling the check
allows for sending workload traffic without encapsulation within the same AWS subnet.
[Default: DoNothing]
enum:
- DoNothing
- Enable
- Disable
type: string
bpfCTLBLogFilter:
description: |-
BPFCTLBLogFilter specifies, what is logged by connect time load balancer when BPFLogLevel is
debug. Currently has to be specified as 'all' when BPFLogFilters is set
to see CTLB logs.
[Default: unset - means logs are emitted when BPFLogLevel id debug and BPFLogFilters not set.]
type: string
bpfConnectTimeLoadBalancing:
description: |-
BPFConnectTimeLoadBalancing when in BPF mode, controls whether Felix installs the connect-time load
balancer. The connect-time load balancer is required for the host to be able to reach Kubernetes services
and it improves the performance of pod-to-service connections.When set to TCP, connect time load balancing
is available only for services with TCP ports. [Default: TCP]
enum:
- TCP
- Enabled
- Disabled
type: string
bpfConnectTimeLoadBalancingEnabled:
description: |-
BPFConnectTimeLoadBalancingEnabled when in BPF mode, controls whether Felix installs the connection-time load
balancer. The connect-time load balancer is required for the host to be able to reach Kubernetes services
and it improves the performance of pod-to-service connections. The only reason to disable it is for debugging
purposes.
Deprecated: Use BPFConnectTimeLoadBalancing [Default: true]
type: boolean
bpfConntrackLogLevel:
description: |-
BPFConntrackLogLevel controls the log level of the BPF conntrack cleanup program, which runs periodically
to clean up expired BPF conntrack entries.
[Default: Off].
enum:
- "Off"
- Debug
type: string
bpfConntrackMode:
description: |-
BPFConntrackCleanupMode controls how BPF conntrack entries are cleaned up. `Auto` will use a BPF program if supported,
falling back to userspace if not. `Userspace` will always use the userspace cleanup code. `BPFProgram` will
always use the BPF program (failing if not supported).
[Default: Auto]
enum:
- Auto
- Userspace
- BPFProgram
type: string
bpfConntrackTimeouts:
description: |-
BPFConntrackTimers overrides the default values for the specified conntrack timer if
set. Each value can be either a duration or `Auto` to pick the value from
a Linux conntrack timeout.
Configurable timers are: CreationGracePeriod, TCPSynSent,
TCPEstablished, TCPFinsSeen, TCPResetSeen, UDPTimeout, GenericTimeout,
ICMPTimeout.
Unset values are replaced by the default values with a warning log for
incorrect values.
properties:
creationGracePeriod:
description: |2-
CreationGracePeriod gives a generic grace period to new connection
before they are considered for cleanup [Default: 10s].
pattern: ^(([0-9]*(\.[0-9]*)?(ms|s|h|m|us)+)+|Auto)$
type: string
genericTimeout:
description: |-
GenericTimeout controls how long it takes before considering this
entry for cleanup after the connection became idle. If set to 'Auto', the
value from nf_conntrack_generic_timeout is used. If nil, Calico uses its
own default value. [Default: 10m].
pattern: ^(([0-9]*(\.[0-9]*)?(ms|s|h|m|us)+)+|Auto)$
type: string
icmpTimeout:
description: |-
ICMPTimeout controls how long it takes before considering this
entry for cleanup after the connection became idle. If set to 'Auto', the
value from nf_conntrack_icmp_timeout is used. If nil, Calico uses its
own default value. [Default: 5s].
pattern: ^(([0-9]*(\.[0-9]*)?(ms|s|h|m|us)+)+|Auto)$
type: string
tcpEstablished:
description: |-
TCPEstablished controls how long it takes before considering this entry for
cleanup after the connection became idle. If set to 'Auto', the
value from nf_conntrack_tcp_timeout_established is used. If nil, Calico uses
its own default value. [Default: 1h].
pattern: ^(([0-9]*(\.[0-9]*)?(ms|s|h|m|us)+)+|Auto)$
type: string
tcpFinsSeen:
description: |-
TCPFinsSeen controls how long it takes before considering this entry for
cleanup after the connection was closed gracefully. If set to 'Auto', the
value from nf_conntrack_tcp_timeout_time_wait is used. If nil, Calico uses
its own default value. [Default: Auto].
pattern: ^(([0-9]*(\.[0-9]*)?(ms|s|h|m|us)+)+|Auto)$
type: string
tcpResetSeen:
description: |-
TCPResetSeen controls how long it takes before considering this entry for
cleanup after the connection was aborted. If nil, Calico uses its own
default value. [Default: 40s].
pattern: ^(([0-9]*(\.[0-9]*)?(ms|s|h|m|us)+)+|Auto)$
type: string
tcpSynSent:
description: |-
TCPSynSent controls how long it takes before considering this entry for
cleanup after the last SYN without a response. If set to 'Auto', the
value from nf_conntrack_tcp_timeout_syn_sent is used. If nil, Calico uses
its own default value. [Default: 20s].
pattern: ^(([0-9]*(\.[0-9]*)?(ms|s|h|m|us)+)+|Auto)$
type: string
udpTimeout:
description: |-
UDPTimeout controls how long it takes before considering this entry for
cleanup after the connection became idle. If nil, Calico uses its own
default value. [Default: 60s].
pattern: ^(([0-9]*(\.[0-9]*)?(ms|s|h|m|us)+)+|Auto)$
type: string
type: object
bpfDSROptoutCIDRs:
description: |-
BPFDSROptoutCIDRs is a list of CIDRs which are excluded from DSR. That is, clients
in those CIDRs will access service node ports as if BPFExternalServiceMode was set to
Tunnel.
items:
type: string
type: array
bpfDataIfacePattern:
description: |-
BPFDataIfacePattern is a regular expression that controls which interfaces Felix should attach BPF programs to
in order to catch traffic to/from the network. This needs to match the interfaces that Calico workload traffic
flows over as well as any interfaces that handle incoming traffic to nodeports and services from outside the
cluster. It should not match the workload interfaces (usually named cali...) or any other special device managed
by Calico itself (e.g., tunnels).
type: string
bpfDisableGROForIfaces:
description: |-
BPFDisableGROForIfaces is a regular expression that controls which interfaces Felix should disable the
Generic Receive Offload [GRO] option. It should not match the workload interfaces (usually named cali...).
type: string
bpfDisableUnprivileged:
description: |-
BPFDisableUnprivileged, if enabled, Felix sets the kernel.unprivileged_bpf_disabled sysctl to disable
unprivileged use of BPF. This ensures that unprivileged users cannot access Calico's BPF maps and
cannot insert their own BPF programs to interfere with Calico's. [Default: true]
type: boolean
bpfEnabled:
description: 'BPFEnabled, if enabled Felix will use the BPF dataplane.
[Default: false]'
type: boolean
bpfEnforceRPF:
description: |-
BPFEnforceRPF enforce strict RPF on all host interfaces with BPF programs regardless of
what is the per-interfaces or global setting. Possible values are Disabled, Strict
or Loose. [Default: Loose]
pattern: ^(?i)(Disabled|Strict|Loose)?$
type: string
bpfExcludeCIDRsFromNAT:
description: |-
BPFExcludeCIDRsFromNAT is a list of CIDRs that are to be excluded from NAT
resolution so that host can handle them. A typical usecase is node local
DNS cache.
items:
type: string
type: array
bpfExportBufferSizeMB:
description: |-
BPFExportBufferSizeMB in BPF mode, controls the buffer size used for sending BPF events to felix.
[Default: 1]
type: integer
bpfExtToServiceConnmark:
description: |-
BPFExtToServiceConnmark in BPF mode, controls a 32bit mark that is set on connections from an
external client to a local service. This mark allows us to control how packets of that
connection are routed within the host and how is routing interpreted by RPF check. [Default: 0]
type: integer
bpfExternalServiceMode:
description: |-
BPFExternalServiceMode in BPF mode, controls how connections from outside the cluster to services (node ports
and cluster IPs) are forwarded to remote workloads. If set to "Tunnel" then both request and response traffic
is tunneled to the remote node. If set to "DSR", the request traffic is tunneled but the response traffic
is sent directly from the remote node. In "DSR" mode, the remote node appears to use the IP of the ingress
node; this requires a permissive L2 network. [Default: Tunnel]
pattern: ^(?i)(Tunnel|DSR)?$
type: string
bpfForceTrackPacketsFromIfaces:
description: |-
BPFForceTrackPacketsFromIfaces in BPF mode, forces traffic from these interfaces
to skip Calico's iptables NOTRACK rule, allowing traffic from those interfaces to be
tracked by Linux conntrack. Should only be used for interfaces that are not used for
the Calico fabric. For example, a docker bridge device for non-Calico-networked
containers. [Default: docker+]
items:
type: string
type: array
bpfHostConntrackBypass:
description: |-
BPFHostConntrackBypass Controls whether to bypass Linux conntrack in BPF mode for
workloads and services. [Default: true - bypass Linux conntrack]
type: boolean
bpfHostNetworkedNATWithoutCTLB:
description: |-
BPFHostNetworkedNATWithoutCTLB when in BPF mode, controls whether Felix does a NAT without CTLB. This along with BPFConnectTimeLoadBalancing
determines the CTLB behavior. [Default: Enabled]
enum:
- Enabled
- Disabled
type: string
bpfKubeProxyEndpointSlicesEnabled:
description: |-
BPFKubeProxyEndpointSlicesEnabled is deprecated and has no effect. BPF
kube-proxy always accepts endpoint slices. This option will be removed in
the next release.
type: boolean
bpfKubeProxyIptablesCleanupEnabled:
description: |-
BPFKubeProxyIptablesCleanupEnabled, if enabled in BPF mode, Felix will proactively clean up the upstream
Kubernetes kube-proxy's iptables chains. Should only be enabled if kube-proxy is not running. [Default: true]
type: boolean
bpfKubeProxyMinSyncPeriod:
description: |-
BPFKubeProxyMinSyncPeriod, in BPF mode, controls the minimum time between updates to the dataplane for Felix's
embedded kube-proxy. Lower values give reduced set-up latency. Higher values reduce Felix CPU usage by
batching up more work. [Default: 1s]
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
bpfL3IfacePattern:
description: |-
BPFL3IfacePattern is a regular expression that allows to list tunnel devices like wireguard or vxlan (i.e., L3 devices)
in addition to BPFDataIfacePattern. That is, tunnel interfaces not created by Calico, that Calico workload traffic flows
over as well as any interfaces that handle incoming traffic to nodeports and services from outside the cluster.
type: string
bpfLogFilters:
additionalProperties:
type: string
description: |-
BPFLogFilters is a map of key=values where the value is
a pcap filter expression and the key is an interface name with 'all'
denoting all interfaces, 'weps' all workload endpoints and 'heps' all host
endpoints.
When specified as an env var, it accepts a comma-separated list of
key=values.
[Default: unset - means all debug logs are emitted]
type: object
bpfLogLevel:
description: |-
BPFLogLevel controls the log level of the BPF programs when in BPF dataplane mode. One of "Off", "Info", or
"Debug". The logs are emitted to the BPF trace pipe, accessible with the command `tc exec bpf debug`.
[Default: Off].
pattern: ^(?i)(Off|Info|Debug)?$
type: string
bpfMapSizeConntrack:
description: |-
BPFMapSizeConntrack sets the size for the conntrack map. This map must be large enough to hold
an entry for each active connection. Warning: changing the size of the conntrack map can cause disruption.
type: integer
bpfMapSizeConntrackCleanupQueue:
description: |-
BPFMapSizeConntrackCleanupQueue sets the size for the map used to hold NAT conntrack entries that are queued
for cleanup. This should be big enough to hold all the NAT entries that expire within one cleanup interval.
minimum: 1
type: integer
bpfMapSizeConntrackScaling:
description: |-
BPFMapSizeConntrackScaling controls whether and how we scale the conntrack map size depending
on its usage. 'Disabled' make the size stay at the default or whatever is set by
BPFMapSizeConntrack*. 'DoubleIfFull' doubles the size when the map is pretty much full even
after cleanups. [Default: DoubleIfFull]
pattern: ^(?i)(Disabled|DoubleIfFull)?$
type: string
bpfMapSizeIPSets:
description: |-
BPFMapSizeIPSets sets the size for ipsets map. The IP sets map must be large enough to hold an entry
for each endpoint matched by every selector in the source/destination matches in network policy. Selectors
such as "all()" can result in large numbers of entries (one entry per endpoint in that case).
type: integer
bpfMapSizeIfState:
description: |-
BPFMapSizeIfState sets the size for ifstate map. The ifstate map must be large enough to hold an entry
for each device (host + workloads) on a host.
type: integer
bpfMapSizeNATAffinity:
description: |-
BPFMapSizeNATAffinity sets the size of the BPF map that stores the affinity of a connection (for services that
enable that feature.
type: integer
bpfMapSizeNATBackend:
description: |-
BPFMapSizeNATBackend sets the size for NAT back end map.
This is the total number of endpoints. This is mostly
more than the size of the number of services.
type: integer
bpfMapSizeNATFrontend:
description: |-
BPFMapSizeNATFrontend sets the size for NAT front end map.
FrontendMap should be large enough to hold an entry for each nodeport,
external IP and each port in each service.
type: integer
bpfMapSizePerCpuConntrack:
description: |-
BPFMapSizePerCPUConntrack determines the size of conntrack map based on the number of CPUs. If set to a
non-zero value, overrides BPFMapSizeConntrack with `BPFMapSizePerCPUConntrack * (Number of CPUs)`.
This map must be large enough to hold an entry for each active connection. Warning: changing the size of the
conntrack map can cause disruption.
type: integer
bpfMapSizeRoute:
description: |-
BPFMapSizeRoute sets the size for the routes map. The routes map should be large enough
to hold one entry per workload and a handful of entries per host (enough to cover its own IPs and
tunnel IPs).
type: integer
bpfPSNATPorts:
anyOf:
- type: integer
- type: string
description: |-
BPFPSNATPorts sets the range from which we randomly pick a port if there is a source port
collision. This should be within the ephemeral range as defined by RFC 6056 (1024–65535) and
preferably outside the ephemeral ranges used by common operating systems. Linux uses
32768–60999, while others mostly use the IANA defined range 49152–65535. It is not necessarily
a problem if this range overlaps with the operating systems. Both ends of the range are
inclusive. [Default: 20000:29999]
pattern: ^.*
x-kubernetes-int-or-string: true
bpfPolicyDebugEnabled:
description: |-
BPFPolicyDebugEnabled when true, Felix records detailed information
about the BPF policy programs, which can be examined with the calico-bpf command-line tool.
type: boolean
bpfProfiling:
description: |-
BPFProfiling controls profiling of BPF programs. At the monent, it can be
Disabled or Enabled. [Default: Disabled]
enum:
- Enabled
- Disabled
type: string
bpfRedirectToPeer:
description: |-
BPFRedirectToPeer controls which whether it is allowed to forward straight to the
peer side of the workload devices. It is allowed for any host L2 devices by default
(L2Only), but it breaks TCP dump on the host side of workload device as it bypasses
it on ingress. Value of Enabled also allows redirection from L3 host devices like
IPIP tunnel or Wireguard directly to the peer side of the workload's device. This
makes redirection faster, however, it breaks tools like tcpdump on the peer side.
Use Enabled with caution. [Default: L2Only]
enum:
- Enabled
- Disabled
- L2Only
type: string
chainInsertMode:
description: |-
ChainInsertMode controls whether Felix hooks the kernel's top-level iptables chains by inserting a rule
at the top of the chain or by appending a rule at the bottom. insert is the safe default since it prevents
Calico's rules from being bypassed. If you switch to append mode, be sure that the other rules in the chains
signal acceptance by falling through to the Calico rules, otherwise the Calico policy will be bypassed.
[Default: insert]
pattern: ^(?i)(Insert|Append)?$
type: string
dataplaneDriver:
description: |-
DataplaneDriver filename of the external dataplane driver to use. Only used if UseInternalDataplaneDriver
is set to false.
type: string
dataplaneWatchdogTimeout:
description: |-
DataplaneWatchdogTimeout is the readiness/liveness timeout used for Felix's (internal) dataplane driver.
Deprecated: replaced by the generic HealthTimeoutOverrides.
type: string
debugDisableLogDropping:
description: |-
DebugDisableLogDropping disables the dropping of log messages when the log buffer is full. This can
significantly impact performance if log write-out is a bottleneck. [Default: false]
type: boolean
debugHost:
description: |-
DebugHost is the host IP or hostname to bind the debug port to. Only used
if DebugPort is set. [Default:localhost]
type: string
debugMemoryProfilePath:
description: DebugMemoryProfilePath is the path to write the memory
profile to when triggered by signal.
type: string
debugPort:
description: |-
DebugPort if set, enables Felix's debug HTTP port, which allows memory and CPU profiles
to be retrieved. The debug port is not secure, it should not be exposed to the internet.
type: integer
debugSimulateCalcGraphHangAfter:
description: |-
DebugSimulateCalcGraphHangAfter is used to simulate a hang in the calculation graph after the specified duration.
This is useful in tests of the watchdog system only!
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
debugSimulateDataplaneApplyDelay:
description: |-
DebugSimulateDataplaneApplyDelay adds an artificial delay to every dataplane operation. This is useful for
simulating a heavily loaded system for test purposes only.
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
debugSimulateDataplaneHangAfter:
description: |-
DebugSimulateDataplaneHangAfter is used to simulate a hang in the dataplane after the specified duration.
This is useful in tests of the watchdog system only!
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
defaultEndpointToHostAction:
description: |-
DefaultEndpointToHostAction controls what happens to traffic that goes from a workload endpoint to the host
itself (after the endpoint's egress policy is applied). By default, Calico blocks traffic from workload
endpoints to the host itself with an iptables "DROP" action. If you want to allow some or all traffic from
endpoint to host, set this parameter to RETURN or ACCEPT. Use RETURN if you have your own rules in the iptables
"INPUT" chain; Calico will insert its rules at the top of that chain, then "RETURN" packets to the "INPUT" chain
once it has completed processing workload endpoint egress policy. Use ACCEPT to unconditionally accept packets
from workloads after processing workload endpoint egress policy. [Default: Drop]
pattern: ^(?i)(Drop|Accept|Return)?$
type: string
deviceRouteProtocol:
description: |-
DeviceRouteProtocol controls the protocol to set on routes programmed by Felix. The protocol is an 8-bit label
used to identify the owner of the route.
type: integer
deviceRouteSourceAddress:
description: |-
DeviceRouteSourceAddress IPv4 address to set as the source hint for routes programmed by Felix. When not set
the source address for local traffic from host to workload will be determined by the kernel.
type: string
deviceRouteSourceAddressIPv6:
description: |-
DeviceRouteSourceAddressIPv6 IPv6 address to set as the source hint for routes programmed by Felix. When not set
the source address for local traffic from host to workload will be determined by the kernel.
type: string
disableConntrackInvalidCheck:
description: |-
DisableConntrackInvalidCheck disables the check for invalid connections in conntrack. While the conntrack
invalid check helps to detect malicious traffic, it can also cause issues with certain multi-NIC scenarios.
type: boolean
endpointReportingDelay:
description: |-
EndpointReportingDelay is the delay before Felix reports endpoint status to the datastore. This is only used
by the OpenStack integration. [Default: 1s]
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
endpointReportingEnabled:
description: |-
EndpointReportingEnabled controls whether Felix reports endpoint status to the datastore. This is only used
by the OpenStack integration. [Default: false]
type: boolean
endpointStatusPathPrefix:
description: |-
EndpointStatusPathPrefix is the path to the directory where endpoint status will be written. Endpoint status
file reporting is disabled if field is left empty.
Chosen directory should match the directory used by the CNI plugin for PodStartupDelay.
[Default: /var/run/calico]
type: string
externalNodesList:
description: |-
ExternalNodesCIDRList is a list of CIDR's of external, non-Calico nodes from which VXLAN/IPIP overlay traffic
will be allowed. By default, external tunneled traffic is blocked to reduce attack surface.
items:
type: string
type: array
failsafeInboundHostPorts:
description: |-
FailsafeInboundHostPorts is a list of ProtoPort struct objects including UDP/TCP/SCTP ports and CIDRs that Felix will
allow incoming traffic to host endpoints on irrespective of the security policy. This is useful to avoid accidentally
cutting off a host with incorrect configuration. For backwards compatibility, if the protocol is not specified,
it defaults to "tcp". If a CIDR is not specified, it will allow traffic from all addresses. To disable all inbound host ports,
use the value "[]". The default value allows ssh access, DHCP, BGP, etcd and the Kubernetes API.
[Default: tcp:22, udp:68, tcp:179, tcp:2379, tcp:2380, tcp:5473, tcp:6443, tcp:6666, tcp:6667 ]
items:
description: ProtoPort is combination of protocol, port, and CIDR.
Protocol and port must be specified.
properties:
net:
type: string
port:
type: integer
protocol:
type: string
required:
- port
type: object
type: array
failsafeOutboundHostPorts:
description: |-
FailsafeOutboundHostPorts is a list of PortProto struct objects including UDP/TCP/SCTP ports and CIDRs that Felix
will allow outgoing traffic from host endpoints to irrespective of the security policy. This is useful to avoid accidentally
cutting off a host with incorrect configuration. For backwards compatibility, if the protocol is not specified, it defaults
to "tcp". If a CIDR is not specified, it will allow traffic from all addresses. To disable all outbound host ports,
use the value "[]". The default value opens etcd's standard ports to ensure that Felix does not get cut off from etcd
as well as allowing DHCP, DNS, BGP and the Kubernetes API.
[Default: udp:53, udp:67, tcp:179, tcp:2379, tcp:2380, tcp:5473, tcp:6443, tcp:6666, tcp:6667 ]
items:
description: ProtoPort is combination of protocol, port, and CIDR.
Protocol and port must be specified.
properties:
net:
type: string
port:
type: integer
protocol:
type: string
required:
- port
type: object
type: array
featureDetectOverride:
description: |-
FeatureDetectOverride is used to override feature detection based on auto-detected platform
capabilities. Values are specified in a comma separated list with no spaces, example;
"SNATFullyRandom=true,MASQFullyRandom=false,RestoreSupportsLock=". A value of "true" or "false" will
force enable/disable feature, empty or omitted values fall back to auto-detection.
pattern: ^([a-zA-Z0-9-_]+=(true|false|),)*([a-zA-Z0-9-_]+=(true|false|))?$
type: string
featureGates:
description: |-
FeatureGates is used to enable or disable tech-preview Calico features.
Values are specified in a comma separated list with no spaces, example;
"BPFConnectTimeLoadBalancingWorkaround=enabled,XyZ=false". This is
used to enable features that are not fully production ready.
pattern: ^([a-zA-Z0-9-_]+=([^=]+),)*([a-zA-Z0-9-_]+=([^=]+))?$
type: string
floatingIPs:
description: |-
FloatingIPs configures whether or not Felix will program non-OpenStack floating IP addresses. (OpenStack-derived
floating IPs are always programmed, regardless of this setting.)
enum:
- Enabled
- Disabled
type: string
flowLogsCollectorDebugTrace:
description: |-
When FlowLogsCollectorDebugTrace is set to true, enables the logs in the collector to be
printed in their entirety.
type: boolean
flowLogsFlushInterval:
description: FlowLogsFlushInterval configures the interval at which
Felix exports flow logs.
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
flowLogsGoldmaneServer:
description: FlowLogGoldmaneServer is the flow server endpoint to
which flow data should be published.
type: string
flowLogsPolicyEvaluationMode:
description: |-
Continuous - Felix evaluates active flows on a regular basis to determine the rule
traces in the flow logs. Any policy updates that impact a flow will be reflected in the
pending_policies field, offering a near-real-time view of policy changes across flows.
None - Felix stops evaluating pending traces.
[Default: Continuous]
enum:
- None
- Continuous
type: string
genericXDPEnabled:
description: |-
GenericXDPEnabled enables Generic XDP so network cards that don't support XDP offload or driver
modes can use XDP. This is not recommended since it doesn't provide better performance than
iptables. [Default: false]
type: boolean
goGCThreshold:
description: |-
GoGCThreshold Sets the Go runtime's garbage collection threshold. I.e. the percentage that the heap is
allowed to grow before garbage collection is triggered. In general, doubling the value halves the CPU time
spent doing GC, but it also doubles peak GC memory overhead. A special value of -1 can be used
to disable GC entirely; this should only be used in conjunction with the GoMemoryLimitMB setting.
This setting is overridden by the GOGC environment variable.
[Default: 40]
type: integer
goMaxProcs:
description: |-
GoMaxProcs sets the maximum number of CPUs that the Go runtime will use concurrently. A value of -1 means
"use the system default"; typically the number of real CPUs on the system.
this setting is overridden by the GOMAXPROCS environment variable.
[Default: -1]
type: integer
goMemoryLimitMB:
description: |-
GoMemoryLimitMB sets a (soft) memory limit for the Go runtime in MB. The Go runtime will try to keep its memory
usage under the limit by triggering GC as needed. To avoid thrashing, it will exceed the limit if GC starts to
take more than 50% of the process's CPU time. A value of -1 disables the memory limit.
Note that the memory limit, if used, must be considerably less than any hard resource limit set at the container
or pod level. This is because felix is not the only process that must run in the container or pod.
This setting is overridden by the GOMEMLIMIT environment variable.
[Default: -1]
type: integer
healthEnabled:
description: |-
HealthEnabled if set to true, enables Felix's health port, which provides readiness and liveness endpoints.
[Default: false]
type: boolean
healthHost:
description: 'HealthHost is the host that the health server should
bind to. [Default: localhost]'
type: string
healthPort:
description: 'HealthPort is the TCP port that the health server should
bind to. [Default: 9099]'
type: integer
healthTimeoutOverrides:
description: |-
HealthTimeoutOverrides allows the internal watchdog timeouts of individual subcomponents to be
overridden. This is useful for working around "false positive" liveness timeouts that can occur
in particularly stressful workloads or if CPU is constrained. For a list of active
subcomponents, see Felix's logs.
items:
properties:
name:
type: string
timeout:
type: string
required:
- name
- timeout
type: object
type: array
interfaceExclude:
description: |-
InterfaceExclude A comma-separated list of interface names that should be excluded when Felix is resolving
host endpoints. The default value ensures that Felix ignores Kubernetes' internal `kube-ipvs0` device. If you
want to exclude multiple interface names using a single value, the list supports regular expressions. For
regular expressions you must wrap the value with `/`. For example having values `/^kube/,veth1` will exclude
all interfaces that begin with `kube` and also the interface `veth1`. [Default: kube-ipvs0]
type: string
interfacePrefix:
description: |-
InterfacePrefix is the interface name prefix that identifies workload endpoints and so distinguishes
them from host endpoint interfaces. Note: in environments other than bare metal, the orchestrators
configure this appropriately. For example our Kubernetes and Docker integrations set the 'cali' value,
and our OpenStack integration sets the 'tap' value. [Default: cali]
type: string
interfaceRefreshInterval:
description: |-
InterfaceRefreshInterval is the period at which Felix rescans local interfaces to verify their state.
The rescan can be disabled by setting the interval to 0.
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
ipForwarding:
description: |-
IPForwarding controls whether Felix sets the host sysctls to enable IP forwarding. IP forwarding is required
when using Calico for workload networking. This should be disabled only on hosts where Calico is used solely for
host protection. In BPF mode, due to a kernel interaction, either IPForwarding must be enabled or BPFEnforceRPF
must be disabled. [Default: Enabled]
enum:
- Enabled
- Disabled
type: string
ipipEnabled:
description: |-
IPIPEnabled overrides whether Felix should configure an IPIP interface on the host. Optional as Felix
determines this based on the existing IP pools. [Default: nil (unset)]
type: boolean
ipipMTU:
description: |-
IPIPMTU controls the MTU to set on the IPIP tunnel device. Optional as Felix auto-detects the MTU based on the
MTU of the host's interfaces. [Default: 0 (auto-detect)]
type: integer
ipsetsRefreshInterval:
description: |-
IpsetsRefreshInterval controls the period at which Felix re-checks all IP sets to look for discrepancies.
Set to 0 to disable the periodic refresh. [Default: 90s]
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
iptablesBackend:
description: |-
IptablesBackend controls which backend of iptables will be used. The default is `Auto`.
Warning: changing this on a running system can leave "orphaned" rules in the "other" backend. These
should be cleaned up to avoid confusing interactions.
pattern: ^(?i)(Auto|Legacy|NFT)?$
type: string
iptablesFilterAllowAction:
description: |-
IptablesFilterAllowAction controls what happens to traffic that is accepted by a Felix policy chain in the
iptables filter table (which is used for "normal" policy). The default will immediately `Accept` the traffic. Use
`Return` to send the traffic back up to the system chains for further processing.
pattern: ^(?i)(Accept|Return)?$
type: string
iptablesFilterDenyAction:
description: |-
IptablesFilterDenyAction controls what happens to traffic that is denied by network policy. By default Calico blocks traffic
with an iptables "DROP" action. If you want to use "REJECT" action instead you can configure it in here.
pattern: ^(?i)(Drop|Reject)?$
type: string
iptablesLockFilePath:
description: |-
IptablesLockFilePath is the location of the iptables lock file. You may need to change this
if the lock file is not in its standard location (for example if you have mapped it into Felix's
container at a different path). [Default: /run/xtables.lock]
type: string
iptablesLockProbeInterval:
description: |-
IptablesLockProbeInterval when IptablesLockTimeout is enabled: the time that Felix will wait between
attempts to acquire the iptables lock if it is not available. Lower values make Felix more
responsive when the lock is contended, but use more CPU. [Default: 50ms]
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
iptablesLockTimeout:
description: |-
IptablesLockTimeout is the time that Felix itself will wait for the iptables lock (rather than delegating the
lock handling to the `iptables` command).
Deprecated: `iptables-restore` v1.8+ always takes the lock, so enabling this feature results in deadlock.
[Default: 0s disabled]
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
iptablesMangleAllowAction:
description: |-
IptablesMangleAllowAction controls what happens to traffic that is accepted by a Felix policy chain in the
iptables mangle table (which is used for "pre-DNAT" policy). The default will immediately `Accept` the traffic.
Use `Return` to send the traffic back up to the system chains for further processing.
pattern: ^(?i)(Accept|Return)?$
type: string
iptablesMarkMask:
description: |-
IptablesMarkMask is the mask that Felix selects its IPTables Mark bits from. Should be a 32 bit hexadecimal
number with at least 8 bits set, none of which clash with any other mark bits in use on the system.
[Default: 0xffff0000]
format: int32
type: integer
iptablesNATOutgoingInterfaceFilter:
description: |-
This parameter can be used to limit the host interfaces on which Calico will apply SNAT to traffic leaving a
Calico IPAM pool with "NAT outgoing" enabled. This can be useful if you have a main data interface, where
traffic should be SNATted and a secondary device (such as the docker bridge) which is local to the host and
doesn't require SNAT. This parameter uses the iptables interface matching syntax, which allows + as a
wildcard. Most users will not need to set this. Example: if your data interfaces are eth0 and eth1 and you
want to exclude the docker bridge, you could set this to eth+
type: string
iptablesPostWriteCheckInterval:
description: |-
IptablesPostWriteCheckInterval is the period after Felix has done a write
to the dataplane that it schedules an extra read back in order to check the write was not
clobbered by another process. This should only occur if another application on the system
doesn't respect the iptables lock. [Default: 1s]
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
iptablesRefreshInterval:
description: |-
IptablesRefreshInterval is the period at which Felix re-checks the IP sets
in the dataplane to ensure that no other process has accidentally broken Calico's rules.
Set to 0 to disable IP sets refresh. Note: the default for this value is lower than the
other refresh intervals as a workaround for a Linux kernel bug that was fixed in kernel
version 4.11. If you are using v4.11 or greater you may want to set this to, a higher value
to reduce Felix CPU usage. [Default: 10s]
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
ipv6Support:
description: IPv6Support controls whether Felix enables support for
IPv6 (if supported by the in-use dataplane).
type: boolean
kubeNodePortRanges:
description: |-
KubeNodePortRanges holds list of port ranges used for service node ports. Only used if felix detects kube-proxy running in ipvs mode.
Felix uses these ranges to separate host and workload traffic. [Default: 30000:32767].
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
logDebugFilenameRegex:
description: |-
LogDebugFilenameRegex controls which source code files have their Debug log output included in the logs.
Only logs from files with names that match the given regular expression are included. The filter only applies
to Debug level logs.
type: string
logFilePath:
description: 'LogFilePath is the full path to the Felix log. Set to
none to disable file logging. [Default: /var/log/calico/felix.log]'
type: string
logPrefix:
description: 'LogPrefix is the log prefix that Felix uses when rendering
LOG rules. [Default: calico-packet]'
type: string
logSeverityFile:
description: 'LogSeverityFile is the log severity above which logs
are sent to the log file. [Default: Info]'
pattern: ^(?i)(Debug|Info|Warning|Error|Fatal)?$
type: string
logSeverityScreen:
description: 'LogSeverityScreen is the log severity above which logs
are sent to the stdout. [Default: Info]'
pattern: ^(?i)(Debug|Info|Warning|Error|Fatal)?$
type: string
logSeveritySys:
description: |-
LogSeveritySys is the log severity above which logs are sent to the syslog. Set to None for no logging to syslog.
[Default: Info]
pattern: ^(?i)(Debug|Info|Warning|Error|Fatal)?$
type: string
maxIpsetSize:
description: |-
MaxIpsetSize is the maximum number of IP addresses that can be stored in an IP set. Not applicable
if using the nftables backend.
type: integer
metadataAddr:
description: |-
MetadataAddr is the IP address or domain name of the server that can answer VM queries for
cloud-init metadata. In OpenStack, this corresponds to the machine running nova-api (or in
Ubuntu, nova-api-metadata). A value of none (case-insensitive) means that Felix should not
set up any NAT rule for the metadata path. [Default: 127.0.0.1]
type: string
metadataPort:
description: |-
MetadataPort is the port of the metadata server. This, combined with global.MetadataAddr (if
not 'None'), is used to set up a NAT rule, from 169.254.169.254:80 to MetadataAddr:MetadataPort.
In most cases this should not need to be changed [Default: 8775].
type: integer
mtuIfacePattern:
description: |-
MTUIfacePattern is a regular expression that controls which interfaces Felix should scan in order
to calculate the host's MTU.
This should not match workload interfaces (usually named cali...).
type: string
natOutgoingAddress:
description: |-
NATOutgoingAddress specifies an address to use when performing source NAT for traffic in a natOutgoing pool that
is leaving the network. By default the address used is an address on the interface the traffic is leaving on
(i.e. it uses the iptables MASQUERADE target).
type: string
natPortRange:
anyOf:
- type: integer
- type: string
description: |-
NATPortRange specifies the range of ports that is used for port mapping when doing outgoing NAT. When unset the default behavior of the
network stack is used.
pattern: ^.*
x-kubernetes-int-or-string: true
netlinkTimeout:
description: |-
NetlinkTimeout is the timeout when talking to the kernel over the netlink protocol, used for programming
routes, rules, and other kernel objects. [Default: 10s]
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
nftablesFilterAllowAction:
description: |-
NftablesFilterAllowAction controls the nftables action that Felix uses to represent the "allow" policy verdict
in the filter table. The default is to `ACCEPT` the traffic, which is a terminal action. Alternatively,
`RETURN` can be used to return the traffic back to the top-level chain for further processing by your rules.
pattern: ^(?i)(Accept|Return)?$
type: string
nftablesFilterDenyAction:
description: |-
NftablesFilterDenyAction controls what happens to traffic that is denied by network policy. By default, Calico
blocks traffic with a "drop" action. If you want to use a "reject" action instead you can configure it here.
pattern: ^(?i)(Drop|Reject)?$
type: string
nftablesMangleAllowAction:
description: |-
NftablesMangleAllowAction controls the nftables action that Felix uses to represent the "allow" policy verdict
in the mangle table. The default is to `ACCEPT` the traffic, which is a terminal action. Alternatively,
`RETURN` can be used to return the traffic back to the top-level chain for further processing by your rules.
pattern: ^(?i)(Accept|Return)?$
type: string
nftablesMarkMask:
description: |-
NftablesMarkMask is the mask that Felix selects its nftables Mark bits from. Should be a 32 bit hexadecimal
number with at least 8 bits set, none of which clash with any other mark bits in use on the system.
[Default: 0xffff0000]
format: int32
type: integer
nftablesMode:
description: 'NFTablesMode configures nftables support in Felix. [Default:
Disabled]'
enum:
- Disabled
- Enabled
- Auto
type: string
nftablesRefreshInterval:
description: 'NftablesRefreshInterval controls the interval at which
Felix periodically refreshes the nftables rules. [Default: 90s]'
type: string
openstackRegion:
description: |-
OpenstackRegion is the name of the region that a particular Felix belongs to. In a multi-region
Calico/OpenStack deployment, this must be configured somehow for each Felix (here in the datamodel,
or in felix.cfg or the environment on each compute node), and must match the [calico]
openstack_region value configured in neutron.conf on each node. [Default: Empty]
type: string
policySyncPathPrefix:
description: |-
PolicySyncPathPrefix is used to by Felix to communicate policy changes to external services,
like Application layer policy. [Default: Empty]
type: string
prometheusGoMetricsEnabled:
description: |-
PrometheusGoMetricsEnabled disables Go runtime metrics collection, which the Prometheus client does by default, when
set to false. This reduces the number of metrics reported, reducing Prometheus load. [Default: true]
type: boolean
prometheusMetricsEnabled:
description: 'PrometheusMetricsEnabled enables the Prometheus metrics
server in Felix if set to true. [Default: false]'
type: boolean
prometheusMetricsHost:
description: 'PrometheusMetricsHost is the host that the Prometheus
metrics server should bind to. [Default: empty]'
type: string
prometheusMetricsPort:
description: 'PrometheusMetricsPort is the TCP port that the Prometheus
metrics server should bind to. [Default: 9091]'
type: integer
prometheusProcessMetricsEnabled:
description: |-
PrometheusProcessMetricsEnabled disables process metrics collection, which the Prometheus client does by default, when
set to false. This reduces the number of metrics reported, reducing Prometheus load. [Default: true]
type: boolean
prometheusWireGuardMetricsEnabled:
description: |-
PrometheusWireGuardMetricsEnabled disables wireguard metrics collection, which the Prometheus client does by default, when
set to false. This reduces the number of metrics reported, reducing Prometheus load. [Default: true]
type: boolean
removeExternalRoutes:
description: |-
RemoveExternalRoutes Controls whether Felix will remove unexpected routes to workload interfaces. Felix will
always clean up expected routes that use the configured DeviceRouteProtocol. To add your own routes, you must
use a distinct protocol (in addition to setting this field to false).
type: boolean
reportingInterval:
description: |-
ReportingInterval is the interval at which Felix reports its status into the datastore or 0 to disable.
Must be non-zero in OpenStack deployments. [Default: 30s]
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
reportingTTL:
description: 'ReportingTTL is the time-to-live setting for process-wide
status reports. [Default: 90s]'
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
routeRefreshInterval:
description: |-
RouteRefreshInterval is the period at which Felix re-checks the routes
in the dataplane to ensure that no other process has accidentally broken Calico's rules.
Set to 0 to disable route refresh. [Default: 90s]
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
routeSource:
description: |-
RouteSource configures where Felix gets its routing information.
- WorkloadIPs: use workload endpoints to construct routes.
- CalicoIPAM: the default - use IPAM data to construct routes.
pattern: ^(?i)(WorkloadIPs|CalicoIPAM)?$
type: string
routeSyncDisabled:
description: |-
RouteSyncDisabled will disable all operations performed on the route table. Set to true to
run in network-policy mode only.
type: boolean
routeTableRange:
description: |-
Deprecated in favor of RouteTableRanges.
Calico programs additional Linux route tables for various purposes.
RouteTableRange specifies the indices of the route tables that Calico should use.
properties:
max:
type: integer
min:
type: integer
required:
- max
- min
type: object
routeTableRanges:
description: |-
Calico programs additional Linux route tables for various purposes.
RouteTableRanges specifies a set of table index ranges that Calico should use.
Deprecates`RouteTableRange`, overrides `RouteTableRange`.
items:
properties:
max:
type: integer
min:
type: integer
required:
- max
- min
type: object
type: array
serviceLoopPrevention:
description: |-
When service IP advertisement is enabled, prevent routing loops to service IPs that are
not in use, by dropping or rejecting packets that do not get DNAT'd by kube-proxy.
Unless set to "Disabled", in which case such routing loops continue to be allowed.
[Default: Drop]
pattern: ^(?i)(Drop|Reject|Disabled)?$
type: string
sidecarAccelerationEnabled:
description: 'SidecarAccelerationEnabled enables experimental sidecar
acceleration [Default: false]'
type: boolean
usageReportingEnabled:
description: |-
UsageReportingEnabled reports anonymous Calico version number and cluster size to projectcalico.org. Logs warnings returned by the usage
server. For example, if a significant security vulnerability has been discovered in the version of Calico being used. [Default: true]
type: boolean
usageReportingInitialDelay:
description: 'UsageReportingInitialDelay controls the minimum delay
before Felix makes a report. [Default: 300s]'
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
usageReportingInterval:
description: 'UsageReportingInterval controls the interval at which
Felix makes reports. [Default: 86400s]'
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
useInternalDataplaneDriver:
description: |-
UseInternalDataplaneDriver, if true, Felix will use its internal dataplane programming logic. If false, it
will launch an external dataplane driver and communicate with it over protobuf.
type: boolean
vxlanEnabled:
description: |-
VXLANEnabled overrides whether Felix should create the VXLAN tunnel device for IPv4 VXLAN networking.
Optional as Felix determines this based on the existing IP pools. [Default: nil (unset)]
type: boolean
vxlanMTU:
description: |-
VXLANMTU is the MTU to set on the IPv4 VXLAN tunnel device. Optional as Felix auto-detects the MTU based on the
MTU of the host's interfaces. [Default: 0 (auto-detect)]
type: integer
vxlanMTUV6:
description: |-
VXLANMTUV6 is the MTU to set on the IPv6 VXLAN tunnel device. Optional as Felix auto-detects the MTU based on the
MTU of the host's interfaces. [Default: 0 (auto-detect)]
type: integer
vxlanPort:
description: 'VXLANPort is the UDP port number to use for VXLAN traffic.
[Default: 4789]'
type: integer
vxlanVNI:
description: |-
VXLANVNI is the VXLAN VNI to use for VXLAN traffic. You may need to change this if the default value is
in use on your system. [Default: 4096]
type: integer
windowsManageFirewallRules:
description: 'WindowsManageFirewallRules configures whether or not
Felix will program Windows Firewall rules (to allow inbound access
to its own metrics ports). [Default: Disabled]'
enum:
- Enabled
- Disabled
type: string
wireguardEnabled:
description: 'WireguardEnabled controls whether Wireguard is enabled
for IPv4 (encapsulating IPv4 traffic over an IPv4 underlay network).
[Default: false]'
type: boolean
wireguardEnabledV6:
description: 'WireguardEnabledV6 controls whether Wireguard is enabled
for IPv6 (encapsulating IPv6 traffic over an IPv6 underlay network).
[Default: false]'
type: boolean
wireguardHostEncryptionEnabled:
description: 'WireguardHostEncryptionEnabled controls whether Wireguard
host-to-host encryption is enabled. [Default: false]'
type: boolean
wireguardInterfaceName:
description: 'WireguardInterfaceName specifies the name to use for
the IPv4 Wireguard interface. [Default: wireguard.cali]'
type: string
wireguardInterfaceNameV6:
description: 'WireguardInterfaceNameV6 specifies the name to use for
the IPv6 Wireguard interface. [Default: wg-v6.cali]'
type: string
wireguardKeepAlive:
description: 'WireguardPersistentKeepAlive controls Wireguard PersistentKeepalive
option. Set 0 to disable. [Default: 0]'
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
wireguardListeningPort:
description: 'WireguardListeningPort controls the listening port used
by IPv4 Wireguard. [Default: 51820]'
type: integer
wireguardListeningPortV6:
description: 'WireguardListeningPortV6 controls the listening port
used by IPv6 Wireguard. [Default: 51821]'
type: integer
wireguardMTU:
description: 'WireguardMTU controls the MTU on the IPv4 Wireguard
interface. See Configuring MTU [Default: 1440]'
type: integer
wireguardMTUV6:
description: 'WireguardMTUV6 controls the MTU on the IPv6 Wireguard
interface. See Configuring MTU [Default: 1420]'
type: integer
wireguardRoutingRulePriority:
description: 'WireguardRoutingRulePriority controls the priority value
to use for the Wireguard routing rule. [Default: 99]'
type: integer
wireguardThreadingEnabled:
description: |-
WireguardThreadingEnabled controls whether Wireguard has Threaded NAPI enabled. [Default: false]
This increases the maximum number of packets a Wireguard interface can process.
Consider threaded NAPI only if you have high packets per second workloads that are causing dropping packets due to a saturated `softirq` CPU core.
There is a [known issue](https://lore.kernel.org/netdev/CALrw=nEoT2emQ0OAYCjM1d_6Xe_kNLSZ6dhjb5FxrLFYh4kozA@mail.gmail.com/T/) with this setting
that may cause NAPI to get stuck holding the global `rtnl_mutex` when a peer is removed.
Workaround: Make sure your Linux kernel [includes this patch](https://github.com/torvalds/linux/commit/56364c910691f6d10ba88c964c9041b9ab777bd6) to unwedge NAPI.
type: boolean
workloadSourceSpoofing:
description: |-
WorkloadSourceSpoofing controls whether pods can use the allowedSourcePrefixes annotation to send traffic with a source IP
address that is not theirs. This is disabled by default. When set to "Any", pods can request any prefix.
pattern: ^(?i)(Disabled|Any)?$
type: string
xdpEnabled:
description: 'XDPEnabled enables XDP acceleration for suitable untracked
incoming deny rules. [Default: true]'
type: boolean
xdpRefreshInterval:
description: |-
XDPRefreshInterval is the period at which Felix re-checks all XDP state to ensure that no
other process has accidentally broken Calico's BPF maps or attached programs. Set to 0 to
disable XDP refresh. [Default: 90s]
pattern: ^([0-9]+(\\.[0-9]+)?(ms|s|m|h))*$
type: string
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: globalnetworkpolicies.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: GlobalNetworkPolicy
listKind: GlobalNetworkPolicyList
plural: globalnetworkpolicies
singular: globalnetworkpolicy
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
properties:
applyOnForward:
description: ApplyOnForward indicates to apply the rules in this policy
on forward traffic.
type: boolean
doNotTrack:
description: |-
DoNotTrack indicates whether packets matched by the rules in this policy should go through
the data plane's connection tracking, such as Linux conntrack. If True, the rules in
this policy are applied before any data plane connection tracking, and packets allowed by
this policy are marked as not to be tracked.
type: boolean
egress:
description: |-
The ordered set of egress rules. Each rule contains a set of packet match criteria and
a corresponding action to apply.
items:
description: |-
A Rule encapsulates a set of match criteria and an action. Both selector-based security Policy
and security Profiles reference rules - separated out as a list of rules for both
ingress and egress packet matching.
Each positive match criteria has a negated version, prefixed with "Not". All the match
criteria within a rule must be satisfied for a packet to match. A single rule can contain
the positive and negative version of a match and both must be satisfied for the rule to match.
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: |-
Methods is an optional field that restricts the rule to apply only to HTTP requests that use one of the listed
HTTP Methods (e.g. GET, PUT, etc.)
Multiple methods are OR'd together.
items:
type: string
type: array
paths:
description: |-
Paths is an optional field that restricts the rule to apply to HTTP requests that use one of the listed
HTTP Paths.
Multiple paths are OR'd together.
e.g:
- exact: /foo
- prefix: /bar
NOTE: Each entry may ONLY specify either a `exact` or a `prefix` match. The validator will check for it.
items:
description: |-
HTTPPath specifies an HTTP path to match. It may be either of the form:
exact: : which matches the path exactly or
prefix: : which matches the path prefix
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
description: |-
ICMP is an optional field that restricts the rule to apply to a specific type and
code of ICMP traffic. This should only be specified if the Protocol field is set to
"ICMP" or "ICMPv6".
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
ipVersion:
description: |-
IPVersion is an optional field that restricts the rule to only match a specific IP
version.
type: integer
metadata:
description: Metadata contains additional information for this
rule
properties:
annotations:
additionalProperties:
type: string
description: Annotations is a set of key value pairs that
give extra information about the rule
type: object
type: object
notICMP:
description: NotICMP is the negated version of the ICMP field.
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
description: NotProtocol is the negated version of the Protocol
field.
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
description: |-
Protocol is an optional field that restricts the rule to only apply to traffic of
a specific IP protocol. Required if any of the EntityRules contain Ports
(because ports only apply to certain protocols).
Must be one of these string values: "TCP", "UDP", "ICMP", "ICMPv6", "SCTP", "UDPLite"
or an integer in the range 1-255.
pattern: ^.*
x-kubernetes-int-or-string: true
source:
description: Source contains the match criteria that apply to
source entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
required:
- action
type: object
type: array
ingress:
description: |-
The ordered set of ingress rules. Each rule contains a set of packet match criteria and
a corresponding action to apply.
items:
description: |-
A Rule encapsulates a set of match criteria and an action. Both selector-based security Policy
and security Profiles reference rules - separated out as a list of rules for both
ingress and egress packet matching.
Each positive match criteria has a negated version, prefixed with "Not". All the match
criteria within a rule must be satisfied for a packet to match. A single rule can contain
the positive and negative version of a match and both must be satisfied for the rule to match.
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: |-
Methods is an optional field that restricts the rule to apply only to HTTP requests that use one of the listed
HTTP Methods (e.g. GET, PUT, etc.)
Multiple methods are OR'd together.
items:
type: string
type: array
paths:
description: |-
Paths is an optional field that restricts the rule to apply to HTTP requests that use one of the listed
HTTP Paths.
Multiple paths are OR'd together.
e.g:
- exact: /foo
- prefix: /bar
NOTE: Each entry may ONLY specify either a `exact` or a `prefix` match. The validator will check for it.
items:
description: |-
HTTPPath specifies an HTTP path to match. It may be either of the form:
exact: : which matches the path exactly or
prefix: : which matches the path prefix
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
description: |-
ICMP is an optional field that restricts the rule to apply to a specific type and
code of ICMP traffic. This should only be specified if the Protocol field is set to
"ICMP" or "ICMPv6".
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
ipVersion:
description: |-
IPVersion is an optional field that restricts the rule to only match a specific IP
version.
type: integer
metadata:
description: Metadata contains additional information for this
rule
properties:
annotations:
additionalProperties:
type: string
description: Annotations is a set of key value pairs that
give extra information about the rule
type: object
type: object
notICMP:
description: NotICMP is the negated version of the ICMP field.
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
description: NotProtocol is the negated version of the Protocol
field.
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
description: |-
Protocol is an optional field that restricts the rule to only apply to traffic of
a specific IP protocol. Required if any of the EntityRules contain Ports
(because ports only apply to certain protocols).
Must be one of these string values: "TCP", "UDP", "ICMP", "ICMPv6", "SCTP", "UDPLite"
or an integer in the range 1-255.
pattern: ^.*
x-kubernetes-int-or-string: true
source:
description: Source contains the match criteria that apply to
source entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
required:
- action
type: object
type: array
namespaceSelector:
description: NamespaceSelector is an optional field for an expression
used to select a pod based on namespaces.
type: string
order:
description: |-
Order is an optional field that specifies the order in which the policy is applied.
Policies with higher "order" are applied after those with lower
order within the same tier. If the order is omitted, it may be considered to be "infinite" - i.e. the
policy will be applied last. Policies with identical order will be applied in
alphanumerical order based on the Policy "Name" within the tier.
type: number
performanceHints:
description: |-
PerformanceHints contains a list of hints to Calico's policy engine to
help process the policy more efficiently. Hints never change the
enforcement behaviour of the policy.
Currently, the only available hint is "AssumeNeededOnEveryNode". When
that hint is set on a policy, Felix will act as if the policy matches
a local endpoint even if it does not. This is useful for "preloading"
any large static policies that are known to be used on every node.
If the policy is _not_ used on a particular node then the work
done to preload the policy (and to maintain it) is wasted.
items:
type: string
type: array
preDNAT:
description: PreDNAT indicates to apply the rules in this policy before
any DNAT.
type: boolean
selector:
description: "The selector is an expression used to pick out the endpoints
that the policy should\nbe applied to.\n\nSelector expressions follow
this syntax:\n\n\tlabel == \"string_literal\" -> comparison, e.g.
my_label == \"foo bar\"\n\tlabel != \"string_literal\" -> not
equal; also matches if label is not present\n\tlabel in { \"a\",
\"b\", \"c\", ... } -> true if the value of label X is one of
\"a\", \"b\", \"c\"\n\tlabel not in { \"a\", \"b\", \"c\", ... }
\ -> true if the value of label X is not one of \"a\", \"b\", \"c\"\n\thas(label_name)
\ -> True if that label is present\n\t! expr -> negation of expr\n\texpr
&& expr -> Short-circuit and\n\texpr || expr -> Short-circuit
or\n\t( expr ) -> parens for grouping\n\tall() or the empty selector
-> matches all endpoints.\n\nLabel names are allowed to contain
alphanumerics, -, _ and /. String literals are more permissive\nbut
they do not support escape characters.\n\nExamples (with made-up
labels):\n\n\ttype == \"webserver\" && deployment == \"prod\"\n\ttype
in {\"frontend\", \"backend\"}\n\tdeployment != \"dev\"\n\t! has(label_name)"
type: string
serviceAccountSelector:
description: ServiceAccountSelector is an optional field for an expression
used to select a pod based on service accounts.
type: string
tier:
description: |-
The name of the tier that this policy belongs to. If this is omitted, the default
tier (name is "default") is assumed. The specified tier must exist in order to create
security policies within the tier, the "default" tier is created automatically if it
does not exist, this means for deployments requiring only a single Tier, the tier name
may be omitted on all policy management requests.
type: string
types:
description: |-
Types indicates whether this policy applies to ingress, or to egress, or to both. When
not explicitly specified (and so the value on creation is empty or nil), Calico defaults
Types according to what Ingress and Egress rules are present in the policy. The
default is:
- [ PolicyTypeIngress ], if there are no Egress rules (including the case where there are
also no Ingress rules)
- [ PolicyTypeEgress ], if there are Egress rules but no Ingress rules
- [ PolicyTypeIngress, PolicyTypeEgress ], if there are both Ingress and Egress rules.
When the policy is read back again, Types will always be one of these values, never empty
or nil.
items:
description: PolicyType enumerates the possible values of the PolicySpec
Types field.
type: string
type: array
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: globalnetworksets.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: GlobalNetworkSet
listKind: GlobalNetworkSetList
plural: globalnetworksets
singular: globalnetworkset
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
description: |-
GlobalNetworkSet contains a set of arbitrary IP sub-networks/CIDRs that share labels to
allow rules to refer to them via selectors. The labels of GlobalNetworkSet are not namespaced.
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: GlobalNetworkSetSpec contains the specification for a NetworkSet
resource.
properties:
nets:
description: The list of IP networks that belong to this set.
items:
type: string
type: array
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: hostendpoints.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: HostEndpoint
listKind: HostEndpointList
plural: hostendpoints
singular: hostendpoint
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: HostEndpointSpec contains the specification for a HostEndpoint
resource.
properties:
expectedIPs:
description: "The expected IP addresses (IPv4 and IPv6) of the endpoint.\nIf
\"InterfaceName\" is not present, Calico will look for an interface
matching any\nof the IPs in the list and apply policy to that.\nNote:\n\tWhen
using the selector match criteria in an ingress or egress security
Policy\n\tor Profile, Calico converts the selector into a set of
IP addresses. For host\n\tendpoints, the ExpectedIPs field is used
for that purpose. (If only the interface\n\tname is specified, Calico
does not learn the IPs of the interface for use in match\n\tcriteria.)"
items:
type: string
type: array
interfaceName:
description: |-
Either "*", or the name of a specific Linux interface to apply policy to; or empty. "*"
indicates that this HostEndpoint governs all traffic to, from or through the default
network namespace of the host named by the "Node" field; entering and leaving that
namespace via any interface, including those from/to non-host-networked local workloads.
If InterfaceName is not "*", this HostEndpoint only governs traffic that enters or leaves
the host through the specific interface named by InterfaceName, or - when InterfaceName
is empty - through the specific interface that has one of the IPs in ExpectedIPs.
Therefore, when InterfaceName is empty, at least one expected IP must be specified. Only
external interfaces (such as "eth0") are supported here; it isn't possible for a
HostEndpoint to protect traffic through a specific local workload interface.
Note: Only some kinds of policy are implemented for "*" HostEndpoints; initially just
pre-DNAT policy. Please check Calico documentation for the latest position.
type: string
node:
description: The node name identifying the Calico node instance.
type: string
ports:
description: Ports contains the endpoint's named ports, which may
be referenced in security policy rules.
items:
properties:
name:
type: string
port:
type: integer
protocol:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
required:
- name
- port
- protocol
type: object
type: array
profiles:
description: |-
A list of identifiers of security Profile objects that apply to this endpoint. Each
profile is applied in the order that they appear in this list. Profile rules are applied
after the selector-based security policy.
items:
type: string
type: array
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: ipamblocks.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPAMBlock
listKind: IPAMBlockList
plural: ipamblocks
singular: ipamblock
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: IPAMBlockSpec contains the specification for an IPAMBlock
resource.
properties:
affinity:
description: |-
Affinity of the block, if this block has one. If set, it will be of the form
"host:". If not set, this block is not affine to a host.
type: string
allocations:
description: |-
Array of allocations in-use within this block. nil entries mean the allocation is free.
For non-nil entries at index i, the index is the ordinal of the allocation within this block
and the value is the index of the associated attributes in the Attributes array.
items:
type: integer
# TODO: This nullable is manually added in. We should update controller-gen
# to handle []*int properly itself.
nullable: true
type: array
attributes:
description: |-
Attributes is an array of arbitrary metadata associated with allocations in the block. To find
attributes for a given allocation, use the value of the allocation's entry in the Allocations array
as the index of the element in this array.
items:
properties:
handle_id:
type: string
secondary:
additionalProperties:
type: string
type: object
type: object
type: array
cidr:
description: The block's CIDR.
type: string
deleted:
description: |-
Deleted is an internal boolean used to workaround a limitation in the Kubernetes API whereby
deletion will not return a conflict error if the block has been updated. It should not be set manually.
type: boolean
sequenceNumber:
default: 0
description: |-
We store a sequence number that is updated each time the block is written.
Each allocation will also store the sequence number of the block at the time of its creation.
When releasing an IP, passing the sequence number associated with the allocation allows us
to protect against a race condition and ensure the IP hasn't been released and re-allocated
since the release request.
format: int64
type: integer
sequenceNumberForAllocation:
additionalProperties:
format: int64
type: integer
description: |-
Map of allocated ordinal within the block to sequence number of the block at
the time of allocation. Kubernetes does not allow numerical keys for maps, so
the key is cast to a string.
type: object
strictAffinity:
description: StrictAffinity on the IPAMBlock is deprecated and no
longer used by the code. Use IPAMConfig StrictAffinity instead.
type: boolean
unallocated:
description: Unallocated is an ordered list of allocations which are
free in the block.
items:
type: integer
type: array
required:
- allocations
- attributes
- cidr
- strictAffinity
- unallocated
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: ipamconfigs.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPAMConfig
listKind: IPAMConfigList
plural: ipamconfigs
singular: ipamconfig
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: IPAMConfigSpec contains the specification for an IPAMConfig
resource.
properties:
autoAllocateBlocks:
type: boolean
maxBlocksPerHost:
description: |-
MaxBlocksPerHost, if non-zero, is the max number of blocks that can be
affine to each host.
maximum: 2147483647
minimum: 0
type: integer
strictAffinity:
type: boolean
required:
- autoAllocateBlocks
- strictAffinity
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: ipamhandles.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPAMHandle
listKind: IPAMHandleList
plural: ipamhandles
singular: ipamhandle
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: IPAMHandleSpec contains the specification for an IPAMHandle
resource.
properties:
block:
additionalProperties:
type: integer
type: object
deleted:
type: boolean
handleID:
type: string
required:
- block
- handleID
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: ippools.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPPool
listKind: IPPoolList
plural: ippools
singular: ippool
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: IPPoolSpec contains the specification for an IPPool resource.
properties:
allowedUses:
description: |-
AllowedUse controls what the IP pool will be used for. If not specified or empty, defaults to
["Tunnel", "Workload"] for back-compatibility
items:
type: string
type: array
assignmentMode:
description: Determines the mode how IP addresses should be assigned
from this pool
enum:
- Automatic
- Manual
type: string
blockSize:
description: The block size to use for IP address assignments from
this pool. Defaults to 26 for IPv4 and 122 for IPv6.
type: integer
cidr:
description: The pool CIDR.
type: string
disableBGPExport:
description: 'Disable exporting routes from this IP Pool''s CIDR over
BGP. [Default: false]'
type: boolean
disabled:
description: When disabled is true, Calico IPAM will not assign addresses
from this pool.
type: boolean
ipip:
description: |-
Deprecated: this field is only used for APIv1 backwards compatibility.
Setting this field is not allowed, this field is for internal use only.
properties:
enabled:
description: |-
When enabled is true, ipip tunneling will be used to deliver packets to
destinations within this pool.
type: boolean
mode:
description: |-
The IPIP mode. This can be one of "always" or "cross-subnet". A mode
of "always" will also use IPIP tunneling for routing to destination IP
addresses within this pool. A mode of "cross-subnet" will only use IPIP
tunneling when the destination node is on a different subnet to the
originating node. The default value (if not specified) is "always".
type: string
type: object
ipipMode:
description: |-
Contains configuration for IPIP tunneling for this pool. If not specified,
then this is defaulted to "Never" (i.e. IPIP tunneling is disabled).
type: string
nat-outgoing:
description: |-
Deprecated: this field is only used for APIv1 backwards compatibility.
Setting this field is not allowed, this field is for internal use only.
type: boolean
natOutgoing:
description: |-
When natOutgoing is true, packets sent from Calico networked containers in
this pool to destinations outside of this pool will be masqueraded.
type: boolean
nodeSelector:
description: Allows IPPool to allocate for a specific node by label
selector.
type: string
vxlanMode:
description: |-
Contains configuration for VXLAN tunneling for this pool. If not specified,
then this is defaulted to "Never" (i.e. VXLAN tunneling is disabled).
type: string
required:
- cidr
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: ipreservations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: IPReservation
listKind: IPReservationList
plural: ipreservations
singular: ipreservation
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: IPReservationSpec contains the specification for an IPReservation
resource.
properties:
reservedCIDRs:
description: ReservedCIDRs is a list of CIDRs and/or IP addresses
that Calico IPAM will exclude from new allocations.
items:
type: string
type: array
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: kubecontrollersconfigurations.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: KubeControllersConfiguration
listKind: KubeControllersConfigurationList
plural: kubecontrollersconfigurations
singular: kubecontrollersconfiguration
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: KubeControllersConfigurationSpec contains the values of the
Kubernetes controllers configuration.
properties:
controllers:
description: Controllers enables and configures individual Kubernetes
controllers
properties:
loadBalancer:
description: LoadBalancer enables and configures the LoadBalancer
controller. Enabled by default, set to nil to disable.
properties:
assignIPs:
type: string
type: object
namespace:
description: Namespace enables and configures the namespace controller.
Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform reconciliation
with the Calico datastore. [Default: 5m]'
type: string
type: object
node:
description: Node enables and configures the node controller.
Enabled by default, set to nil to disable.
properties:
hostEndpoint:
description: HostEndpoint controls syncing nodes to host endpoints.
Disabled by default, set to nil to disable.
properties:
autoCreate:
description: 'AutoCreate enables automatic creation of
host endpoints for every node. [Default: Disabled]'
type: string
createDefaultHostEndpoint:
type: string
templates:
description: Templates contains definition for creating
AutoHostEndpoints
items:
properties:
generateName:
description: GenerateName is appended to the end
of the generated AutoHostEndpoint name
type: string
interfaceCIDRs:
description: InterfaceCIDRs contains a list of CIRDs
used for matching nodeIPs to the AutoHostEndpoint
items:
type: string
type: array
labels:
additionalProperties:
type: string
description: Labels adds the specified labels to
the generated AutoHostEndpoint, labels from node
with the same name will be overwritten by values
from the template label
type: object
nodeSelector:
description: NodeSelector allows the AutoHostEndpoint
to be created only for specific nodes
type: string
type: object
type: array
type: object
leakGracePeriod:
description: |-
LeakGracePeriod is the period used by the controller to determine if an IP address has been leaked.
Set to 0 to disable IP garbage collection. [Default: 15m]
type: string
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform reconciliation
with the Calico datastore. [Default: 5m]'
type: string
syncLabels:
description: 'SyncLabels controls whether to copy Kubernetes
node labels to Calico nodes. [Default: Enabled]'
type: string
type: object
policy:
description: Policy enables and configures the policy controller.
Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform reconciliation
with the Calico datastore. [Default: 5m]'
type: string
type: object
serviceAccount:
description: ServiceAccount enables and configures the service
account controller. Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform reconciliation
with the Calico datastore. [Default: 5m]'
type: string
type: object
workloadEndpoint:
description: WorkloadEndpoint enables and configures the workload
endpoint controller. Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform reconciliation
with the Calico datastore. [Default: 5m]'
type: string
type: object
type: object
debugProfilePort:
description: |-
DebugProfilePort configures the port to serve memory and cpu profiles on. If not specified, profiling
is disabled.
format: int32
type: integer
etcdV3CompactionPeriod:
description: 'EtcdV3CompactionPeriod is the period between etcdv3
compaction requests. Set to 0 to disable. [Default: 10m]'
type: string
healthChecks:
description: 'HealthChecks enables or disables support for health
checks [Default: Enabled]'
type: string
logSeverityScreen:
description: 'LogSeverityScreen is the log severity above which logs
are sent to the stdout. [Default: Info]'
type: string
prometheusMetricsPort:
description: 'PrometheusMetricsPort is the TCP port that the Prometheus
metrics server should bind to. Set to 0 to disable. [Default: 9094]'
type: integer
required:
- controllers
type: object
status:
description: |-
KubeControllersConfigurationStatus represents the status of the configuration. It's useful for admins to
be able to see the actual config that was applied, which can be modified by environment variables on the
kube-controllers process.
properties:
environmentVars:
additionalProperties:
type: string
description: |-
EnvironmentVars contains the environment variables on the kube-controllers that influenced
the RunningConfig.
type: object
runningConfig:
description: |-
RunningConfig contains the effective config that is running in the kube-controllers pod, after
merging the API resource with any environment variables.
properties:
controllers:
description: Controllers enables and configures individual Kubernetes
controllers
properties:
loadBalancer:
description: LoadBalancer enables and configures the LoadBalancer
controller. Enabled by default, set to nil to disable.
properties:
assignIPs:
type: string
type: object
namespace:
description: Namespace enables and configures the namespace
controller. Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform
reconciliation with the Calico datastore. [Default:
5m]'
type: string
type: object
node:
description: Node enables and configures the node controller.
Enabled by default, set to nil to disable.
properties:
hostEndpoint:
description: HostEndpoint controls syncing nodes to host
endpoints. Disabled by default, set to nil to disable.
properties:
autoCreate:
description: 'AutoCreate enables automatic creation
of host endpoints for every node. [Default: Disabled]'
type: string
createDefaultHostEndpoint:
type: string
templates:
description: Templates contains definition for creating
AutoHostEndpoints
items:
properties:
generateName:
description: GenerateName is appended to the
end of the generated AutoHostEndpoint name
type: string
interfaceCIDRs:
description: InterfaceCIDRs contains a list
of CIRDs used for matching nodeIPs to the
AutoHostEndpoint
items:
type: string
type: array
labels:
additionalProperties:
type: string
description: Labels adds the specified labels
to the generated AutoHostEndpoint, labels
from node with the same name will be overwritten
by values from the template label
type: object
nodeSelector:
description: NodeSelector allows the AutoHostEndpoint
to be created only for specific nodes
type: string
type: object
type: array
type: object
leakGracePeriod:
description: |-
LeakGracePeriod is the period used by the controller to determine if an IP address has been leaked.
Set to 0 to disable IP garbage collection. [Default: 15m]
type: string
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform
reconciliation with the Calico datastore. [Default:
5m]'
type: string
syncLabels:
description: 'SyncLabels controls whether to copy Kubernetes
node labels to Calico nodes. [Default: Enabled]'
type: string
type: object
policy:
description: Policy enables and configures the policy controller.
Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform
reconciliation with the Calico datastore. [Default:
5m]'
type: string
type: object
serviceAccount:
description: ServiceAccount enables and configures the service
account controller. Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform
reconciliation with the Calico datastore. [Default:
5m]'
type: string
type: object
workloadEndpoint:
description: WorkloadEndpoint enables and configures the workload
endpoint controller. Enabled by default, set to nil to disable.
properties:
reconcilerPeriod:
description: 'ReconcilerPeriod is the period to perform
reconciliation with the Calico datastore. [Default:
5m]'
type: string
type: object
type: object
debugProfilePort:
description: |-
DebugProfilePort configures the port to serve memory and cpu profiles on. If not specified, profiling
is disabled.
format: int32
type: integer
etcdV3CompactionPeriod:
description: 'EtcdV3CompactionPeriod is the period between etcdv3
compaction requests. Set to 0 to disable. [Default: 10m]'
type: string
healthChecks:
description: 'HealthChecks enables or disables support for health
checks [Default: Enabled]'
type: string
logSeverityScreen:
description: 'LogSeverityScreen is the log severity above which
logs are sent to the stdout. [Default: Info]'
type: string
prometheusMetricsPort:
description: 'PrometheusMetricsPort is the TCP port that the Prometheus
metrics server should bind to. Set to 0 to disable. [Default:
9094]'
type: integer
required:
- controllers
type: object
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: networkpolicies.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: NetworkPolicy
listKind: NetworkPolicyList
plural: networkpolicies
singular: networkpolicy
preserveUnknownFields: false
scope: Namespaced
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
properties:
egress:
description: |-
The ordered set of egress rules. Each rule contains a set of packet match criteria and
a corresponding action to apply.
items:
description: |-
A Rule encapsulates a set of match criteria and an action. Both selector-based security Policy
and security Profiles reference rules - separated out as a list of rules for both
ingress and egress packet matching.
Each positive match criteria has a negated version, prefixed with "Not". All the match
criteria within a rule must be satisfied for a packet to match. A single rule can contain
the positive and negative version of a match and both must be satisfied for the rule to match.
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: |-
Methods is an optional field that restricts the rule to apply only to HTTP requests that use one of the listed
HTTP Methods (e.g. GET, PUT, etc.)
Multiple methods are OR'd together.
items:
type: string
type: array
paths:
description: |-
Paths is an optional field that restricts the rule to apply to HTTP requests that use one of the listed
HTTP Paths.
Multiple paths are OR'd together.
e.g:
- exact: /foo
- prefix: /bar
NOTE: Each entry may ONLY specify either a `exact` or a `prefix` match. The validator will check for it.
items:
description: |-
HTTPPath specifies an HTTP path to match. It may be either of the form:
exact: : which matches the path exactly or
prefix: : which matches the path prefix
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
description: |-
ICMP is an optional field that restricts the rule to apply to a specific type and
code of ICMP traffic. This should only be specified if the Protocol field is set to
"ICMP" or "ICMPv6".
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
ipVersion:
description: |-
IPVersion is an optional field that restricts the rule to only match a specific IP
version.
type: integer
metadata:
description: Metadata contains additional information for this
rule
properties:
annotations:
additionalProperties:
type: string
description: Annotations is a set of key value pairs that
give extra information about the rule
type: object
type: object
notICMP:
description: NotICMP is the negated version of the ICMP field.
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
description: NotProtocol is the negated version of the Protocol
field.
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
description: |-
Protocol is an optional field that restricts the rule to only apply to traffic of
a specific IP protocol. Required if any of the EntityRules contain Ports
(because ports only apply to certain protocols).
Must be one of these string values: "TCP", "UDP", "ICMP", "ICMPv6", "SCTP", "UDPLite"
or an integer in the range 1-255.
pattern: ^.*
x-kubernetes-int-or-string: true
source:
description: Source contains the match criteria that apply to
source entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
required:
- action
type: object
type: array
ingress:
description: |-
The ordered set of ingress rules. Each rule contains a set of packet match criteria and
a corresponding action to apply.
items:
description: |-
A Rule encapsulates a set of match criteria and an action. Both selector-based security Policy
and security Profiles reference rules - separated out as a list of rules for both
ingress and egress packet matching.
Each positive match criteria has a negated version, prefixed with "Not". All the match
criteria within a rule must be satisfied for a packet to match. A single rule can contain
the positive and negative version of a match and both must be satisfied for the rule to match.
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: |-
Methods is an optional field that restricts the rule to apply only to HTTP requests that use one of the listed
HTTP Methods (e.g. GET, PUT, etc.)
Multiple methods are OR'd together.
items:
type: string
type: array
paths:
description: |-
Paths is an optional field that restricts the rule to apply to HTTP requests that use one of the listed
HTTP Paths.
Multiple paths are OR'd together.
e.g:
- exact: /foo
- prefix: /bar
NOTE: Each entry may ONLY specify either a `exact` or a `prefix` match. The validator will check for it.
items:
description: |-
HTTPPath specifies an HTTP path to match. It may be either of the form:
exact: : which matches the path exactly or
prefix: : which matches the path prefix
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
description: |-
ICMP is an optional field that restricts the rule to apply to a specific type and
code of ICMP traffic. This should only be specified if the Protocol field is set to
"ICMP" or "ICMPv6".
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
ipVersion:
description: |-
IPVersion is an optional field that restricts the rule to only match a specific IP
version.
type: integer
metadata:
description: Metadata contains additional information for this
rule
properties:
annotations:
additionalProperties:
type: string
description: Annotations is a set of key value pairs that
give extra information about the rule
type: object
type: object
notICMP:
description: NotICMP is the negated version of the ICMP field.
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
description: NotProtocol is the negated version of the Protocol
field.
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
description: |-
Protocol is an optional field that restricts the rule to only apply to traffic of
a specific IP protocol. Required if any of the EntityRules contain Ports
(because ports only apply to certain protocols).
Must be one of these string values: "TCP", "UDP", "ICMP", "ICMPv6", "SCTP", "UDPLite"
or an integer in the range 1-255.
pattern: ^.*
x-kubernetes-int-or-string: true
source:
description: Source contains the match criteria that apply to
source entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
required:
- action
type: object
type: array
order:
description: |-
Order is an optional field that specifies the order in which the policy is applied.
Policies with higher "order" are applied after those with lower
order within the same tier. If the order is omitted, it may be considered to be "infinite" - i.e. the
policy will be applied last. Policies with identical order will be applied in
alphanumerical order based on the Policy "Name" within the tier.
type: number
performanceHints:
description: |-
PerformanceHints contains a list of hints to Calico's policy engine to
help process the policy more efficiently. Hints never change the
enforcement behaviour of the policy.
Currently, the only available hint is "AssumeNeededOnEveryNode". When
that hint is set on a policy, Felix will act as if the policy matches
a local endpoint even if it does not. This is useful for "preloading"
any large static policies that are known to be used on every node.
If the policy is _not_ used on a particular node then the work
done to preload the policy (and to maintain it) is wasted.
items:
type: string
type: array
selector:
description: "The selector is an expression used to pick out the endpoints
that the policy should\nbe applied to.\n\nSelector expressions follow
this syntax:\n\n\tlabel == \"string_literal\" -> comparison, e.g.
my_label == \"foo bar\"\n\tlabel != \"string_literal\" -> not
equal; also matches if label is not present\n\tlabel in { \"a\",
\"b\", \"c\", ... } -> true if the value of label X is one of
\"a\", \"b\", \"c\"\n\tlabel not in { \"a\", \"b\", \"c\", ... }
\ -> true if the value of label X is not one of \"a\", \"b\", \"c\"\n\thas(label_name)
\ -> True if that label is present\n\t! expr -> negation of expr\n\texpr
&& expr -> Short-circuit and\n\texpr || expr -> Short-circuit
or\n\t( expr ) -> parens for grouping\n\tall() or the empty selector
-> matches all endpoints.\n\nLabel names are allowed to contain
alphanumerics, -, _ and /. String literals are more permissive\nbut
they do not support escape characters.\n\nExamples (with made-up
labels):\n\n\ttype == \"webserver\" && deployment == \"prod\"\n\ttype
in {\"frontend\", \"backend\"}\n\tdeployment != \"dev\"\n\t! has(label_name)"
type: string
serviceAccountSelector:
description: ServiceAccountSelector is an optional field for an expression
used to select a pod based on service accounts.
type: string
tier:
description: |-
The name of the tier that this policy belongs to. If this is omitted, the default
tier (name is "default") is assumed. The specified tier must exist in order to create
security policies within the tier, the "default" tier is created automatically if it
does not exist, this means for deployments requiring only a single Tier, the tier name
may be omitted on all policy management requests.
type: string
types:
description: |-
Types indicates whether this policy applies to ingress, or to egress, or to both. When
not explicitly specified (and so the value on creation is empty or nil), Calico defaults
Types according to what Ingress and Egress are present in the policy. The
default is:
- [ PolicyTypeIngress ], if there are no Egress rules (including the case where there are
also no Ingress rules)
- [ PolicyTypeEgress ], if there are Egress rules but no Ingress rules
- [ PolicyTypeIngress, PolicyTypeEgress ], if there are both Ingress and Egress rules.
When the policy is read back again, Types will always be one of these values, never empty
or nil.
items:
description: PolicyType enumerates the possible values of the PolicySpec
Types field.
type: string
type: array
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: networksets.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: NetworkSet
listKind: NetworkSetList
plural: networksets
singular: networkset
preserveUnknownFields: false
scope: Namespaced
versions:
- name: v1
schema:
openAPIV3Schema:
description: NetworkSet is the Namespaced-equivalent of the GlobalNetworkSet.
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: NetworkSetSpec contains the specification for a NetworkSet
resource.
properties:
nets:
description: The list of IP networks that belong to this set.
items:
type: string
type: array
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: stagedglobalnetworkpolicies.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: StagedGlobalNetworkPolicy
listKind: StagedGlobalNetworkPolicyList
plural: stagedglobalnetworkpolicies
singular: stagedglobalnetworkpolicy
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
properties:
applyOnForward:
description: ApplyOnForward indicates to apply the rules in this policy
on forward traffic.
type: boolean
doNotTrack:
description: |-
DoNotTrack indicates whether packets matched by the rules in this policy should go through
the data plane's connection tracking, such as Linux conntrack. If True, the rules in
this policy are applied before any data plane connection tracking, and packets allowed by
this policy are marked as not to be tracked.
type: boolean
egress:
description: |-
The ordered set of egress rules. Each rule contains a set of packet match criteria and
a corresponding action to apply.
items:
description: |-
A Rule encapsulates a set of match criteria and an action. Both selector-based security Policy
and security Profiles reference rules - separated out as a list of rules for both
ingress and egress packet matching.
Each positive match criteria has a negated version, prefixed with "Not". All the match
criteria within a rule must be satisfied for a packet to match. A single rule can contain
the positive and negative version of a match and both must be satisfied for the rule to match.
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: |-
Methods is an optional field that restricts the rule to apply only to HTTP requests that use one of the listed
HTTP Methods (e.g. GET, PUT, etc.)
Multiple methods are OR'd together.
items:
type: string
type: array
paths:
description: |-
Paths is an optional field that restricts the rule to apply to HTTP requests that use one of the listed
HTTP Paths.
Multiple paths are OR'd together.
e.g:
- exact: /foo
- prefix: /bar
NOTE: Each entry may ONLY specify either a `exact` or a `prefix` match. The validator will check for it.
items:
description: |-
HTTPPath specifies an HTTP path to match. It may be either of the form:
exact: : which matches the path exactly or
prefix: : which matches the path prefix
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
description: |-
ICMP is an optional field that restricts the rule to apply to a specific type and
code of ICMP traffic. This should only be specified if the Protocol field is set to
"ICMP" or "ICMPv6".
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
ipVersion:
description: |-
IPVersion is an optional field that restricts the rule to only match a specific IP
version.
type: integer
metadata:
description: Metadata contains additional information for this
rule
properties:
annotations:
additionalProperties:
type: string
description: Annotations is a set of key value pairs that
give extra information about the rule
type: object
type: object
notICMP:
description: NotICMP is the negated version of the ICMP field.
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
description: NotProtocol is the negated version of the Protocol
field.
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
description: |-
Protocol is an optional field that restricts the rule to only apply to traffic of
a specific IP protocol. Required if any of the EntityRules contain Ports
(because ports only apply to certain protocols).
Must be one of these string values: "TCP", "UDP", "ICMP", "ICMPv6", "SCTP", "UDPLite"
or an integer in the range 1-255.
pattern: ^.*
x-kubernetes-int-or-string: true
source:
description: Source contains the match criteria that apply to
source entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
required:
- action
type: object
type: array
ingress:
description: |-
The ordered set of ingress rules. Each rule contains a set of packet match criteria and
a corresponding action to apply.
items:
description: |-
A Rule encapsulates a set of match criteria and an action. Both selector-based security Policy
and security Profiles reference rules - separated out as a list of rules for both
ingress and egress packet matching.
Each positive match criteria has a negated version, prefixed with "Not". All the match
criteria within a rule must be satisfied for a packet to match. A single rule can contain
the positive and negative version of a match and both must be satisfied for the rule to match.
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: |-
Methods is an optional field that restricts the rule to apply only to HTTP requests that use one of the listed
HTTP Methods (e.g. GET, PUT, etc.)
Multiple methods are OR'd together.
items:
type: string
type: array
paths:
description: |-
Paths is an optional field that restricts the rule to apply to HTTP requests that use one of the listed
HTTP Paths.
Multiple paths are OR'd together.
e.g:
- exact: /foo
- prefix: /bar
NOTE: Each entry may ONLY specify either a `exact` or a `prefix` match. The validator will check for it.
items:
description: |-
HTTPPath specifies an HTTP path to match. It may be either of the form:
exact: : which matches the path exactly or
prefix: : which matches the path prefix
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
description: |-
ICMP is an optional field that restricts the rule to apply to a specific type and
code of ICMP traffic. This should only be specified if the Protocol field is set to
"ICMP" or "ICMPv6".
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
ipVersion:
description: |-
IPVersion is an optional field that restricts the rule to only match a specific IP
version.
type: integer
metadata:
description: Metadata contains additional information for this
rule
properties:
annotations:
additionalProperties:
type: string
description: Annotations is a set of key value pairs that
give extra information about the rule
type: object
type: object
notICMP:
description: NotICMP is the negated version of the ICMP field.
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
description: NotProtocol is the negated version of the Protocol
field.
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
description: |-
Protocol is an optional field that restricts the rule to only apply to traffic of
a specific IP protocol. Required if any of the EntityRules contain Ports
(because ports only apply to certain protocols).
Must be one of these string values: "TCP", "UDP", "ICMP", "ICMPv6", "SCTP", "UDPLite"
or an integer in the range 1-255.
pattern: ^.*
x-kubernetes-int-or-string: true
source:
description: Source contains the match criteria that apply to
source entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
required:
- action
type: object
type: array
namespaceSelector:
description: NamespaceSelector is an optional field for an expression
used to select a pod based on namespaces.
type: string
order:
description: |-
Order is an optional field that specifies the order in which the policy is applied.
Policies with higher "order" are applied after those with lower
order within the same tier. If the order is omitted, it may be considered to be "infinite" - i.e. the
policy will be applied last. Policies with identical order will be applied in
alphanumerical order based on the Policy "Name" within the tier.
type: number
performanceHints:
description: |-
PerformanceHints contains a list of hints to Calico's policy engine to
help process the policy more efficiently. Hints never change the
enforcement behaviour of the policy.
Currently, the only available hint is "AssumeNeededOnEveryNode". When
that hint is set on a policy, Felix will act as if the policy matches
a local endpoint even if it does not. This is useful for "preloading"
any large static policies that are known to be used on every node.
If the policy is _not_ used on a particular node then the work
done to preload the policy (and to maintain it) is wasted.
items:
type: string
type: array
preDNAT:
description: PreDNAT indicates to apply the rules in this policy before
any DNAT.
type: boolean
selector:
description: "The selector is an expression used to pick pick out
the endpoints that the policy should\nbe applied to.\n\nSelector
expressions follow this syntax:\n\n\tlabel == \"string_literal\"
\ -> comparison, e.g. my_label == \"foo bar\"\n\tlabel != \"string_literal\"
\ -> not equal; also matches if label is not present\n\tlabel
in { \"a\", \"b\", \"c\", ... } -> true if the value of label
X is one of \"a\", \"b\", \"c\"\n\tlabel not in { \"a\", \"b\",
\"c\", ... } -> true if the value of label X is not one of \"a\",
\"b\", \"c\"\n\thas(label_name) -> True if that label is present\n\t!
expr -> negation of expr\n\texpr && expr -> Short-circuit and\n\texpr
|| expr -> Short-circuit or\n\t( expr ) -> parens for grouping\n\tall()
or the empty selector -> matches all endpoints.\n\nLabel names are
allowed to contain alphanumerics, -, _ and /. String literals are
more permissive\nbut they do not support escape characters.\n\nExamples
(with made-up labels):\n\n\ttype == \"webserver\" && deployment
== \"prod\"\n\ttype in {\"frontend\", \"backend\"}\n\tdeployment
!= \"dev\"\n\t! has(label_name)"
type: string
serviceAccountSelector:
description: ServiceAccountSelector is an optional field for an expression
used to select a pod based on service accounts.
type: string
stagedAction:
description: The staged action. If this is omitted, the default is
Set.
type: string
tier:
description: |-
The name of the tier that this policy belongs to. If this is omitted, the default
tier (name is "default") is assumed. The specified tier must exist in order to create
security policies within the tier, the "default" tier is created automatically if it
does not exist, this means for deployments requiring only a single Tier, the tier name
may be omitted on all policy management requests.
type: string
types:
description: |-
Types indicates whether this policy applies to ingress, or to egress, or to both. When
not explicitly specified (and so the value on creation is empty or nil), Calico defaults
Types according to what Ingress and Egress rules are present in the policy. The
default is:
- [ PolicyTypeIngress ], if there are no Egress rules (including the case where there are
also no Ingress rules)
- [ PolicyTypeEgress ], if there are Egress rules but no Ingress rules
- [ PolicyTypeIngress, PolicyTypeEgress ], if there are both Ingress and Egress rules.
When the policy is read back again, Types will always be one of these values, never empty
or nil.
items:
description: PolicyType enumerates the possible values of the PolicySpec
Types field.
type: string
type: array
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: stagedkubernetesnetworkpolicies.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: StagedKubernetesNetworkPolicy
listKind: StagedKubernetesNetworkPolicyList
plural: stagedkubernetesnetworkpolicies
singular: stagedkubernetesnetworkpolicy
preserveUnknownFields: false
scope: Namespaced
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
properties:
egress:
description: |-
List of egress rules to be applied to the selected pods. Outgoing traffic is
allowed if there are no NetworkPolicies selecting the pod (and cluster policy
otherwise allows the traffic), OR if the traffic matches at least one egress rule
across all of the NetworkPolicy objects whose podSelector matches the pod. If
this field is empty then this NetworkPolicy limits all outgoing traffic (and serves
solely to ensure that the pods it selects are isolated by default).
This field is beta-level in 1.8
items:
description: |-
NetworkPolicyEgressRule describes a particular set of traffic that is allowed out of pods
matched by a NetworkPolicySpec's podSelector. The traffic must match both ports and to.
This type is beta-level in 1.8
properties:
ports:
description: |-
ports is a list of destination ports for outgoing traffic.
Each item in this list is combined using a logical OR. If this field is
empty or missing, this rule matches all ports (traffic not restricted by port).
If this field is present and contains at least one item, then this rule allows
traffic only if the traffic matches at least one port in the list.
items:
description: NetworkPolicyPort describes a port to allow traffic
on
properties:
endPort:
description: |-
endPort indicates that the range of ports from port to endPort if set, inclusive,
should be allowed by the policy. This field cannot be defined if the port field
is not defined or if the port field is defined as a named (string) port.
The endPort must be equal or greater than port.
format: int32
type: integer
port:
anyOf:
- type: integer
- type: string
description: |-
port represents the port on the given protocol. This can either be a numerical or named
port on a pod. If this field is not provided, this matches all port names and
numbers.
If present, only traffic on the specified protocol AND port will be matched.
x-kubernetes-int-or-string: true
protocol:
description: |-
protocol represents the protocol (TCP, UDP, or SCTP) which traffic must match.
If not specified, this field defaults to TCP.
type: string
type: object
type: array
x-kubernetes-list-type: atomic
to:
description: |-
to is a list of destinations for outgoing traffic of pods selected for this rule.
Items in this list are combined using a logical OR operation. If this field is
empty or missing, this rule matches all destinations (traffic not restricted by
destination). If this field is present and contains at least one item, this rule
allows traffic only if the traffic matches at least one item in the to list.
items:
description: |-
NetworkPolicyPeer describes a peer to allow traffic to/from. Only certain combinations of
fields are allowed
properties:
ipBlock:
description: |-
ipBlock defines policy on a particular IPBlock. If this field is set then
neither of the other fields can be.
properties:
cidr:
description: |-
cidr is a string representing the IPBlock
Valid examples are "192.168.1.0/24" or "2001:db8::/64"
type: string
except:
description: |-
except is a slice of CIDRs that should not be included within an IPBlock
Valid examples are "192.168.1.0/24" or "2001:db8::/64"
Except values will be rejected if they are outside the cidr range
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- cidr
type: object
namespaceSelector:
description: |-
namespaceSelector selects namespaces using cluster-scoped labels. This field follows
standard label selector semantics; if present but empty, it selects all namespaces.
If podSelector is also set, then the NetworkPolicyPeer as a whole selects
the pods matching podSelector in the namespaces selected by namespaceSelector.
Otherwise it selects all pods in the namespaces selected by namespaceSelector.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
podSelector:
description: |-
podSelector is a label selector which selects pods. This field follows standard label
selector semantics; if present but empty, it selects all pods.
If namespaceSelector is also set, then the NetworkPolicyPeer as a whole selects
the pods matching podSelector in the Namespaces selected by NamespaceSelector.
Otherwise it selects the pods matching podSelector in the policy's own namespace.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
type: object
type: array
x-kubernetes-list-type: atomic
type: object
type: array
ingress:
description: |-
List of ingress rules to be applied to the selected pods. Traffic is allowed to
a pod if there are no NetworkPolicies selecting the pod
(and cluster policy otherwise allows the traffic), OR if the traffic source is
the pod's local node, OR if the traffic matches at least one ingress rule
across all of the NetworkPolicy objects whose podSelector matches the pod. If
this field is empty then this NetworkPolicy does not allow any traffic (and serves
solely to ensure that the pods it selects are isolated by default)
items:
description: |-
NetworkPolicyIngressRule describes a particular set of traffic that is allowed to the pods
matched by a NetworkPolicySpec's podSelector. The traffic must match both ports and from.
properties:
from:
description: |-
from is a list of sources which should be able to access the pods selected for this rule.
Items in this list are combined using a logical OR operation. If this field is
empty or missing, this rule matches all sources (traffic not restricted by
source). If this field is present and contains at least one item, this rule
allows traffic only if the traffic matches at least one item in the from list.
items:
description: |-
NetworkPolicyPeer describes a peer to allow traffic to/from. Only certain combinations of
fields are allowed
properties:
ipBlock:
description: |-
ipBlock defines policy on a particular IPBlock. If this field is set then
neither of the other fields can be.
properties:
cidr:
description: |-
cidr is a string representing the IPBlock
Valid examples are "192.168.1.0/24" or "2001:db8::/64"
type: string
except:
description: |-
except is a slice of CIDRs that should not be included within an IPBlock
Valid examples are "192.168.1.0/24" or "2001:db8::/64"
Except values will be rejected if they are outside the cidr range
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- cidr
type: object
namespaceSelector:
description: |-
namespaceSelector selects namespaces using cluster-scoped labels. This field follows
standard label selector semantics; if present but empty, it selects all namespaces.
If podSelector is also set, then the NetworkPolicyPeer as a whole selects
the pods matching podSelector in the namespaces selected by namespaceSelector.
Otherwise it selects all pods in the namespaces selected by namespaceSelector.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
podSelector:
description: |-
podSelector is a label selector which selects pods. This field follows standard label
selector semantics; if present but empty, it selects all pods.
If namespaceSelector is also set, then the NetworkPolicyPeer as a whole selects
the pods matching podSelector in the Namespaces selected by NamespaceSelector.
Otherwise it selects the pods matching podSelector in the policy's own namespace.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
type: object
type: array
x-kubernetes-list-type: atomic
ports:
description: |-
ports is a list of ports which should be made accessible on the pods selected for
this rule. Each item in this list is combined using a logical OR. If this field is
empty or missing, this rule matches all ports (traffic not restricted by port).
If this field is present and contains at least one item, then this rule allows
traffic only if the traffic matches at least one port in the list.
items:
description: NetworkPolicyPort describes a port to allow traffic
on
properties:
endPort:
description: |-
endPort indicates that the range of ports from port to endPort if set, inclusive,
should be allowed by the policy. This field cannot be defined if the port field
is not defined or if the port field is defined as a named (string) port.
The endPort must be equal or greater than port.
format: int32
type: integer
port:
anyOf:
- type: integer
- type: string
description: |-
port represents the port on the given protocol. This can either be a numerical or named
port on a pod. If this field is not provided, this matches all port names and
numbers.
If present, only traffic on the specified protocol AND port will be matched.
x-kubernetes-int-or-string: true
protocol:
description: |-
protocol represents the protocol (TCP, UDP, or SCTP) which traffic must match.
If not specified, this field defaults to TCP.
type: string
type: object
type: array
x-kubernetes-list-type: atomic
type: object
type: array
podSelector:
description: |-
Selects the pods to which this NetworkPolicy object applies. The array of
ingress rules is applied to any pods selected by this field. Multiple network
policies can select the same set of pods. In this case, the ingress rules for
each are combined additively. This field is NOT optional and follows standard
label selector semantics. An empty podSelector matches all pods in this
namespace.
properties:
matchExpressions:
description: matchExpressions is a list of label selector requirements.
The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector applies
to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
x-kubernetes-list-type: atomic
required:
- key
- operator
type: object
type: array
x-kubernetes-list-type: atomic
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
policyTypes:
description: |-
List of rule types that the NetworkPolicy relates to.
Valid options are Ingress, Egress, or Ingress,Egress.
If this field is not specified, it will default based on the existence of Ingress or Egress rules;
policies that contain an Egress section are assumed to affect Egress, and all policies
(whether or not they contain an Ingress section) are assumed to affect Ingress.
If you want to write an egress-only policy, you must explicitly specify policyTypes [ "Egress" ].
Likewise, if you want to write a policy that specifies that no egress is allowed,
you must specify a policyTypes value that include "Egress" (since such a policy would not include
an Egress section and would otherwise default to just [ "Ingress" ]).
This field is beta-level in 1.8
items:
description: |-
PolicyType string describes the NetworkPolicy type
This type is beta-level in 1.8
type: string
type: array
stagedAction:
description: The staged action. If this is omitted, the default is
Set.
type: string
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: stagednetworkpolicies.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: StagedNetworkPolicy
listKind: StagedNetworkPolicyList
plural: stagednetworkpolicies
singular: stagednetworkpolicy
preserveUnknownFields: false
scope: Namespaced
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
properties:
egress:
description: |-
The ordered set of egress rules. Each rule contains a set of packet match criteria and
a corresponding action to apply.
items:
description: |-
A Rule encapsulates a set of match criteria and an action. Both selector-based security Policy
and security Profiles reference rules - separated out as a list of rules for both
ingress and egress packet matching.
Each positive match criteria has a negated version, prefixed with "Not". All the match
criteria within a rule must be satisfied for a packet to match. A single rule can contain
the positive and negative version of a match and both must be satisfied for the rule to match.
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: |-
Methods is an optional field that restricts the rule to apply only to HTTP requests that use one of the listed
HTTP Methods (e.g. GET, PUT, etc.)
Multiple methods are OR'd together.
items:
type: string
type: array
paths:
description: |-
Paths is an optional field that restricts the rule to apply to HTTP requests that use one of the listed
HTTP Paths.
Multiple paths are OR'd together.
e.g:
- exact: /foo
- prefix: /bar
NOTE: Each entry may ONLY specify either a `exact` or a `prefix` match. The validator will check for it.
items:
description: |-
HTTPPath specifies an HTTP path to match. It may be either of the form:
exact: : which matches the path exactly or
prefix: : which matches the path prefix
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
description: |-
ICMP is an optional field that restricts the rule to apply to a specific type and
code of ICMP traffic. This should only be specified if the Protocol field is set to
"ICMP" or "ICMPv6".
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
ipVersion:
description: |-
IPVersion is an optional field that restricts the rule to only match a specific IP
version.
type: integer
metadata:
description: Metadata contains additional information for this
rule
properties:
annotations:
additionalProperties:
type: string
description: Annotations is a set of key value pairs that
give extra information about the rule
type: object
type: object
notICMP:
description: NotICMP is the negated version of the ICMP field.
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
description: NotProtocol is the negated version of the Protocol
field.
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
description: |-
Protocol is an optional field that restricts the rule to only apply to traffic of
a specific IP protocol. Required if any of the EntityRules contain Ports
(because ports only apply to certain protocols).
Must be one of these string values: "TCP", "UDP", "ICMP", "ICMPv6", "SCTP", "UDPLite"
or an integer in the range 1-255.
pattern: ^.*
x-kubernetes-int-or-string: true
source:
description: Source contains the match criteria that apply to
source entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
required:
- action
type: object
type: array
ingress:
description: |-
The ordered set of ingress rules. Each rule contains a set of packet match criteria and
a corresponding action to apply.
items:
description: |-
A Rule encapsulates a set of match criteria and an action. Both selector-based security Policy
and security Profiles reference rules - separated out as a list of rules for both
ingress and egress packet matching.
Each positive match criteria has a negated version, prefixed with "Not". All the match
criteria within a rule must be satisfied for a packet to match. A single rule can contain
the positive and negative version of a match and both must be satisfied for the rule to match.
properties:
action:
type: string
destination:
description: Destination contains the match criteria that apply
to destination entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
http:
description: HTTP contains match criteria that apply to HTTP
requests.
properties:
methods:
description: |-
Methods is an optional field that restricts the rule to apply only to HTTP requests that use one of the listed
HTTP Methods (e.g. GET, PUT, etc.)
Multiple methods are OR'd together.
items:
type: string
type: array
paths:
description: |-
Paths is an optional field that restricts the rule to apply to HTTP requests that use one of the listed
HTTP Paths.
Multiple paths are OR'd together.
e.g:
- exact: /foo
- prefix: /bar
NOTE: Each entry may ONLY specify either a `exact` or a `prefix` match. The validator will check for it.
items:
description: |-
HTTPPath specifies an HTTP path to match. It may be either of the form:
exact: : which matches the path exactly or
prefix: : which matches the path prefix
properties:
exact:
type: string
prefix:
type: string
type: object
type: array
type: object
icmp:
description: |-
ICMP is an optional field that restricts the rule to apply to a specific type and
code of ICMP traffic. This should only be specified if the Protocol field is set to
"ICMP" or "ICMPv6".
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
ipVersion:
description: |-
IPVersion is an optional field that restricts the rule to only match a specific IP
version.
type: integer
metadata:
description: Metadata contains additional information for this
rule
properties:
annotations:
additionalProperties:
type: string
description: Annotations is a set of key value pairs that
give extra information about the rule
type: object
type: object
notICMP:
description: NotICMP is the negated version of the ICMP field.
properties:
code:
description: |-
Match on a specific ICMP code. If specified, the Type value must also be specified.
This is a technical limitation imposed by the kernel's iptables firewall, which
Calico uses to enforce the rule.
type: integer
type:
description: |-
Match on a specific ICMP type. For example a value of 8 refers to ICMP Echo Request
(i.e. pings).
type: integer
type: object
notProtocol:
anyOf:
- type: integer
- type: string
description: NotProtocol is the negated version of the Protocol
field.
pattern: ^.*
x-kubernetes-int-or-string: true
protocol:
anyOf:
- type: integer
- type: string
description: |-
Protocol is an optional field that restricts the rule to only apply to traffic of
a specific IP protocol. Required if any of the EntityRules contain Ports
(because ports only apply to certain protocols).
Must be one of these string values: "TCP", "UDP", "ICMP", "ICMPv6", "SCTP", "UDPLite"
or an integer in the range 1-255.
pattern: ^.*
x-kubernetes-int-or-string: true
source:
description: Source contains the match criteria that apply to
source entity.
properties:
namespaceSelector:
description: |-
NamespaceSelector is an optional field that contains a selector expression. Only traffic
that originates from (or terminates at) endpoints within the selected namespaces will be
matched. When both NamespaceSelector and another selector are defined on the same rule, then only
workload endpoints that are matched by both selectors will be selected by the rule.
For NetworkPolicy, an empty NamespaceSelector implies that the Selector is limited to selecting
only workload endpoints in the same namespace as the NetworkPolicy.
For NetworkPolicy, `global()` NamespaceSelector implies that the Selector is limited to selecting
only GlobalNetworkSet or HostEndpoint.
For GlobalNetworkPolicy, an empty NamespaceSelector implies the Selector applies to workload
endpoints across all namespaces.
type: string
nets:
description: |-
Nets is an optional field that restricts the rule to only apply to traffic that
originates from (or terminates at) IP addresses in any of the given subnets.
items:
type: string
type: array
notNets:
description: NotNets is the negated version of the Nets
field.
items:
type: string
type: array
notPorts:
description: |-
NotPorts is the negated version of the Ports field.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
notSelector:
description: |-
NotSelector is the negated version of the Selector field. See Selector field for
subtleties with negated selectors.
type: string
ports:
description: |-
Ports is an optional field that restricts the rule to only apply to traffic that has a
source (destination) port that matches one of these ranges/values. This value is a
list of integers or strings that represent ranges of ports.
Since only some protocols have ports, if any ports are specified it requires the
Protocol match in the Rule to be set to "TCP" or "UDP".
items:
anyOf:
- type: integer
- type: string
pattern: ^.*
x-kubernetes-int-or-string: true
type: array
selector:
description: "Selector is an optional field that contains
a selector expression (see Policy for\nsample syntax).
\ Only traffic that originates from (terminates at) endpoints
matching\nthe selector will be matched.\n\nNote that:
in addition to the negated version of the Selector (see
NotSelector below), the\nselector expression syntax itself
supports negation. The two types of negation are subtly\ndifferent.
One negates the set of matched endpoints, the other negates
the whole match:\n\n\tSelector = \"!has(my_label)\" matches
packets that are from other Calico-controlled\n\tendpoints
that do not have the label \"my_label\".\n\n\tNotSelector
= \"has(my_label)\" matches packets that are not from
Calico-controlled\n\tendpoints that do have the label
\"my_label\".\n\nThe effect is that the latter will accept
packets from non-Calico sources whereas the\nformer is
limited to packets from Calico-controlled endpoints."
type: string
serviceAccounts:
description: |-
ServiceAccounts is an optional field that restricts the rule to only apply to traffic that originates from (or
terminates at) a pod running as a matching service account.
properties:
names:
description: |-
Names is an optional field that restricts the rule to only apply to traffic that originates from (or terminates
at) a pod running as a service account whose name is in the list.
items:
type: string
type: array
selector:
description: |-
Selector is an optional field that restricts the rule to only apply to traffic that originates from
(or terminates at) a pod running as a service account that matches the given label selector.
If both Names and Selector are specified then they are AND'ed.
type: string
type: object
services:
description: |-
Services is an optional field that contains options for matching Kubernetes Services.
If specified, only traffic that originates from or terminates at endpoints within the selected
service(s) will be matched, and only to/from each endpoint's port.
Services cannot be specified on the same rule as Selector, NotSelector, NamespaceSelector, Nets,
NotNets or ServiceAccounts.
Ports and NotPorts can only be specified with Services on ingress rules.
properties:
name:
description: Name specifies the name of a Kubernetes
Service to match.
type: string
namespace:
description: |-
Namespace specifies the namespace of the given Service. If left empty, the rule
will match within this policy's namespace.
type: string
type: object
type: object
required:
- action
type: object
type: array
order:
description: |-
Order is an optional field that specifies the order in which the policy is applied.
Policies with higher "order" are applied after those with lower
order within the same tier. If the order is omitted, it may be considered to be "infinite" - i.e. the
policy will be applied last. Policies with identical order will be applied in
alphanumerical order based on the Policy "Name" within the tier.
type: number
performanceHints:
description: |-
PerformanceHints contains a list of hints to Calico's policy engine to
help process the policy more efficiently. Hints never change the
enforcement behaviour of the policy.
Currently, the only available hint is "AssumeNeededOnEveryNode". When
that hint is set on a policy, Felix will act as if the policy matches
a local endpoint even if it does not. This is useful for "preloading"
any large static policies that are known to be used on every node.
If the policy is _not_ used on a particular node then the work
done to preload the policy (and to maintain it) is wasted.
items:
type: string
type: array
selector:
description: "The selector is an expression used to pick pick out
the endpoints that the policy should\nbe applied to.\n\nSelector
expressions follow this syntax:\n\n\tlabel == \"string_literal\"
\ -> comparison, e.g. my_label == \"foo bar\"\n\tlabel != \"string_literal\"
\ -> not equal; also matches if label is not present\n\tlabel
in { \"a\", \"b\", \"c\", ... } -> true if the value of label
X is one of \"a\", \"b\", \"c\"\n\tlabel not in { \"a\", \"b\",
\"c\", ... } -> true if the value of label X is not one of \"a\",
\"b\", \"c\"\n\thas(label_name) -> True if that label is present\n\t!
expr -> negation of expr\n\texpr && expr -> Short-circuit and\n\texpr
|| expr -> Short-circuit or\n\t( expr ) -> parens for grouping\n\tall()
or the empty selector -> matches all endpoints.\n\nLabel names are
allowed to contain alphanumerics, -, _ and /. String literals are
more permissive\nbut they do not support escape characters.\n\nExamples
(with made-up labels):\n\n\ttype == \"webserver\" && deployment
== \"prod\"\n\ttype in {\"frontend\", \"backend\"}\n\tdeployment
!= \"dev\"\n\t! has(label_name)"
type: string
serviceAccountSelector:
description: ServiceAccountSelector is an optional field for an expression
used to select a pod based on service accounts.
type: string
stagedAction:
description: The staged action. If this is omitted, the default is
Set.
type: string
tier:
description: |-
The name of the tier that this policy belongs to. If this is omitted, the default
tier (name is "default") is assumed. The specified tier must exist in order to create
security policies within the tier, the "default" tier is created automatically if it
does not exist, this means for deployments requiring only a single Tier, the tier name
may be omitted on all policy management requests.
type: string
types:
description: |-
Types indicates whether this policy applies to ingress, or to egress, or to both. When
not explicitly specified (and so the value on creation is empty or nil), Calico defaults
Types according to what Ingress and Egress are present in the policy. The
default is:
- [ PolicyTypeIngress ], if there are no Egress rules (including the case where there are
also no Ingress rules)
- [ PolicyTypeEgress ], if there are Egress rules but no Ingress rules
- [ PolicyTypeIngress, PolicyTypeEgress ], if there are both Ingress and Egress rules.
When the policy is read back again, Types will always be one of these values, never empty
or nil.
items:
description: PolicyType enumerates the possible values of the PolicySpec
Types field.
type: string
type: array
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.17.3
name: tiers.crd.projectcalico.org
spec:
group: crd.projectcalico.org
names:
kind: Tier
listKind: TierList
plural: tiers
singular: tier
preserveUnknownFields: false
scope: Cluster
versions:
- name: v1
schema:
openAPIV3Schema:
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: TierSpec contains the specification for a security policy
tier resource.
properties:
defaultAction:
description: |-
DefaultAction specifies the action applied to workloads selected by a policy in the tier,
but not rule matched the workload's traffic.
[Default: Deny]
enum:
- Pass
- Deny
type: string
order:
description: |-
Order is an optional field that specifies the order in which the tier is applied.
Tiers with higher "order" are applied after those with lower order. If the order
is omitted, it may be considered to be "infinite" - i.e. the tier will be applied
last. Tiers with identical order will be applied in alphanumerical order based
on the Tier "Name".
type: number
type: object
type: object
served: true
storage: true
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
api-approved.kubernetes.io: https://github.com/kubernetes-sigs/network-policy-api/pull/30
policy.networking.k8s.io/bundle-version: v0.1.1
policy.networking.k8s.io/channel: experimental
creationTimestamp: null
name: adminnetworkpolicies.policy.networking.k8s.io
spec:
group: policy.networking.k8s.io
names:
kind: AdminNetworkPolicy
listKind: AdminNetworkPolicyList
plural: adminnetworkpolicies
shortNames:
- anp
singular: adminnetworkpolicy
scope: Cluster
versions:
- additionalPrinterColumns:
- jsonPath: .spec.priority
name: Priority
type: string
- jsonPath: .metadata.creationTimestamp
name: Age
type: date
name: v1alpha1
schema:
openAPIV3Schema:
description: |-
AdminNetworkPolicy is a cluster level resource that is part of the
AdminNetworkPolicy API.
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: Specification of the desired behavior of AdminNetworkPolicy.
properties:
egress:
description: |-
Egress is the list of Egress rules to be applied to the selected pods.
A total of 100 rules will be allowed in each ANP instance.
The relative precedence of egress rules within a single ANP object (all of
which share the priority) will be determined by the order in which the rule
is written. Thus, a rule that appears at the top of the egress rules
would take the highest precedence.
ANPs with no egress rules do not affect egress traffic.
Support: Core
items:
description: |-
AdminNetworkPolicyEgressRule describes an action to take on a particular
set of traffic originating from pods selected by a AdminNetworkPolicy's
Subject field.
properties:
action:
description: |-
Action specifies the effect this rule will have on matching traffic.
Currently the following actions are supported:
Allow: allows the selected traffic (even if it would otherwise have been denied by NetworkPolicy)
Deny: denies the selected traffic
Pass: instructs the selected traffic to skip any remaining ANP rules, and
then pass execution to any NetworkPolicies that select the pod.
If the pod is not selected by any NetworkPolicies then execution
is passed to any BaselineAdminNetworkPolicies that select the pod.
Support: Core
enum:
- Allow
- Deny
- Pass
type: string
name:
description: |-
Name is an identifier for this rule, that may be no more than 100 characters
in length. This field should be used by the implementation to help
improve observability, readability and error-reporting for any applied
AdminNetworkPolicies.
Support: Core
maxLength: 100
type: string
ports:
description: |-
Ports allows for matching traffic based on port and protocols.
This field is a list of destination ports for the outgoing egress traffic.
If Ports is not set then the rule does not filter traffic via port.
Support: Core
items:
description: |-
AdminNetworkPolicyPort describes how to select network ports on pod(s).
Exactly one field must be set.
maxProperties: 1
minProperties: 1
properties:
namedPort:
description: |-
NamedPort selects a port on a pod(s) based on name.
Support: Extended
type: string
portNumber:
description: |-
Port selects a port on a pod(s) based on number.
Support: Core
properties:
port:
description: |-
Number defines a network port value.
Support: Core
format: int32
maximum: 65535
minimum: 1
type: integer
protocol:
default: TCP
description: |-
Protocol is the network protocol (TCP, UDP, or SCTP) which traffic must
match. If not specified, this field defaults to TCP.
Support: Core
type: string
required:
- port
- protocol
type: object
portRange:
description: |-
PortRange selects a port range on a pod(s) based on provided start and end
values.
Support: Core
properties:
end:
description: |-
End defines a network port that is the end of a port range, the End value
must be greater than Start.
Support: Core
format: int32
maximum: 65535
minimum: 1
type: integer
protocol:
default: TCP
description: |-
Protocol is the network protocol (TCP, UDP, or SCTP) which traffic must
match. If not specified, this field defaults to TCP.
Support: Core
type: string
start:
description: |-
Start defines a network port that is the start of a port range, the Start
value must be less than End.
Support: Core
format: int32
maximum: 65535
minimum: 1
type: integer
required:
- end
- start
type: object
type: object
maxItems: 100
type: array
to:
description: |-
To is the List of destinations whose traffic this rule applies to.
If any AdminNetworkPolicyEgressPeer matches the destination of outgoing
traffic then the specified action is applied.
This field must be defined and contain at least one item.
Support: Core
items:
description: |-
AdminNetworkPolicyEgressPeer defines a peer to allow traffic to.
Exactly one of the selector pointers must be set for a given peer. If a
consumer observes none of its fields are set, they must assume an unknown
option has been specified and fail closed.
maxProperties: 1
minProperties: 1
properties:
namespaces:
description: |-
Namespaces defines a way to select all pods within a set of Namespaces.
Note that host-networked pods are not included in this type of peer.
Support: Core
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
networks:
description: |-
Networks defines a way to select peers via CIDR blocks.
This is intended for representing entities that live outside the cluster,
which can't be selected by pods, namespaces and nodes peers, but note
that cluster-internal traffic will be checked against the rule as
well. So if you Allow or Deny traffic to `"0.0.0.0/0"`, that will allow
or deny all IPv4 pod-to-pod traffic as well. If you don't want that,
add a rule that Passes all pod traffic before the Networks rule.
Each item in Networks should be provided in the CIDR format and should be
IPv4 or IPv6, for example "10.0.0.0/8" or "fd00::/8".
Networks can have upto 25 CIDRs specified.
Support: Extended
items:
description: |-
CIDR is an IP address range in CIDR notation (for example, "10.0.0.0/8" or "fd00::/8").
This string must be validated by implementations using net.ParseCIDR
TODO: Introduce CEL CIDR validation regex isCIDR() in Kube 1.31 when it is available.
maxLength: 43
type: string
x-kubernetes-validations:
- message: CIDR must be either an IPv4 or IPv6 address.
IPv4 address embedded in IPv6 addresses are not
supported
rule: self.contains(':') != self.contains('.')
maxItems: 25
minItems: 1
type: array
x-kubernetes-list-type: set
nodes:
description: |-
Nodes defines a way to select a set of nodes in
the cluster. This field follows standard label selector
semantics; if present but empty, it selects all Nodes.
Support: Extended
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
pods:
description: |-
Pods defines a way to select a set of pods in
a set of namespaces. Note that host-networked pods
are not included in this type of peer.
Support: Core
properties:
namespaceSelector:
description: |-
NamespaceSelector follows standard label selector semantics; if empty,
it selects all Namespaces.
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the
selector applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
podSelector:
description: |-
PodSelector is used to explicitly select pods within a namespace; if empty,
it selects all Pods.
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the
selector applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
required:
- namespaceSelector
- podSelector
type: object
type: object
maxItems: 100
minItems: 1
type: array
required:
- action
- to
type: object
x-kubernetes-validations:
- message: networks/nodes peer cannot be set with namedPorts since
there are no namedPorts for networks/nodes
rule: '!(self.to.exists(peer, has(peer.networks) || has(peer.nodes))
&& has(self.ports) && self.ports.exists(port, has(port.namedPort)))'
maxItems: 100
type: array
ingress:
description: |-
Ingress is the list of Ingress rules to be applied to the selected pods.
A total of 100 rules will be allowed in each ANP instance.
The relative precedence of ingress rules within a single ANP object (all of
which share the priority) will be determined by the order in which the rule
is written. Thus, a rule that appears at the top of the ingress rules
would take the highest precedence.
ANPs with no ingress rules do not affect ingress traffic.
Support: Core
items:
description: |-
AdminNetworkPolicyIngressRule describes an action to take on a particular
set of traffic destined for pods selected by an AdminNetworkPolicy's
Subject field.
properties:
action:
description: |-
Action specifies the effect this rule will have on matching traffic.
Currently the following actions are supported:
Allow: allows the selected traffic (even if it would otherwise have been denied by NetworkPolicy)
Deny: denies the selected traffic
Pass: instructs the selected traffic to skip any remaining ANP rules, and
then pass execution to any NetworkPolicies that select the pod.
If the pod is not selected by any NetworkPolicies then execution
is passed to any BaselineAdminNetworkPolicies that select the pod.
Support: Core
enum:
- Allow
- Deny
- Pass
type: string
from:
description: |-
From is the list of sources whose traffic this rule applies to.
If any AdminNetworkPolicyIngressPeer matches the source of incoming
traffic then the specified action is applied.
This field must be defined and contain at least one item.
Support: Core
items:
description: |-
AdminNetworkPolicyIngressPeer defines an in-cluster peer to allow traffic from.
Exactly one of the selector pointers must be set for a given peer. If a
consumer observes none of its fields are set, they must assume an unknown
option has been specified and fail closed.
maxProperties: 1
minProperties: 1
properties:
namespaces:
description: |-
Namespaces defines a way to select all pods within a set of Namespaces.
Note that host-networked pods are not included in this type of peer.
Support: Core
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
pods:
description: |-
Pods defines a way to select a set of pods in
a set of namespaces. Note that host-networked pods
are not included in this type of peer.
Support: Core
properties:
namespaceSelector:
description: |-
NamespaceSelector follows standard label selector semantics; if empty,
it selects all Namespaces.
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the
selector applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
podSelector:
description: |-
PodSelector is used to explicitly select pods within a namespace; if empty,
it selects all Pods.
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the
selector applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
required:
- namespaceSelector
- podSelector
type: object
type: object
maxItems: 100
minItems: 1
type: array
name:
description: |-
Name is an identifier for this rule, that may be no more than 100 characters
in length. This field should be used by the implementation to help
improve observability, readability and error-reporting for any applied
AdminNetworkPolicies.
Support: Core
maxLength: 100
type: string
ports:
description: |-
Ports allows for matching traffic based on port and protocols.
This field is a list of ports which should be matched on
the pods selected for this policy i.e the subject of the policy.
So it matches on the destination port for the ingress traffic.
If Ports is not set then the rule does not filter traffic via port.
Support: Core
items:
description: |-
AdminNetworkPolicyPort describes how to select network ports on pod(s).
Exactly one field must be set.
maxProperties: 1
minProperties: 1
properties:
namedPort:
description: |-
NamedPort selects a port on a pod(s) based on name.
Support: Extended
type: string
portNumber:
description: |-
Port selects a port on a pod(s) based on number.
Support: Core
properties:
port:
description: |-
Number defines a network port value.
Support: Core
format: int32
maximum: 65535
minimum: 1
type: integer
protocol:
default: TCP
description: |-
Protocol is the network protocol (TCP, UDP, or SCTP) which traffic must
match. If not specified, this field defaults to TCP.
Support: Core
type: string
required:
- port
- protocol
type: object
portRange:
description: |-
PortRange selects a port range on a pod(s) based on provided start and end
values.
Support: Core
properties:
end:
description: |-
End defines a network port that is the end of a port range, the End value
must be greater than Start.
Support: Core
format: int32
maximum: 65535
minimum: 1
type: integer
protocol:
default: TCP
description: |-
Protocol is the network protocol (TCP, UDP, or SCTP) which traffic must
match. If not specified, this field defaults to TCP.
Support: Core
type: string
start:
description: |-
Start defines a network port that is the start of a port range, the Start
value must be less than End.
Support: Core
format: int32
maximum: 65535
minimum: 1
type: integer
required:
- end
- start
type: object
type: object
maxItems: 100
type: array
required:
- action
- from
type: object
maxItems: 100
type: array
priority:
description: |-
Priority is a value from 0 to 1000. Rules with lower priority values have
higher precedence, and are checked before rules with higher priority values.
All AdminNetworkPolicy rules have higher precedence than NetworkPolicy or
BaselineAdminNetworkPolicy rules
The behavior is undefined if two ANP objects have same priority.
Support: Core
format: int32
maximum: 1000
minimum: 0
type: integer
subject:
description: |-
Subject defines the pods to which this AdminNetworkPolicy applies.
Note that host-networked pods are not included in subject selection.
Support: Core
maxProperties: 1
minProperties: 1
properties:
namespaces:
description: Namespaces is used to select pods via namespace selectors.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
pods:
description: Pods is used to select pods via namespace AND pod
selectors.
properties:
namespaceSelector:
description: |-
NamespaceSelector follows standard label selector semantics; if empty,
it selects all Namespaces.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
podSelector:
description: |-
PodSelector is used to explicitly select pods within a namespace; if empty,
it selects all Pods.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
required:
- namespaceSelector
- podSelector
type: object
type: object
required:
- priority
- subject
type: object
status:
description: Status is the status to be reported by the implementation.
properties:
conditions:
items:
description: "Condition contains details for one aspect of the current
state of this API Resource.\n---\nThis struct is intended for
direct use as an array at the field path .status.conditions. For
example,\n\n\n\ttype FooStatus struct{\n\t // Represents the
observations of a foo's current state.\n\t // Known .status.conditions.type
are: \"Available\", \"Progressing\", and \"Degraded\"\n\t //
+patchMergeKey=type\n\t // +patchStrategy=merge\n\t // +listType=map\n\t
\ // +listMapKey=type\n\t Conditions []metav1.Condition `json:\"conditions,omitempty\"
patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"`\n\n\n\t
\ // other fields\n\t}"
properties:
lastTransitionTime:
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format: date-time
type: string
message:
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength: 32768
type: string
observedGeneration:
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format: int64
minimum: 0
type: integer
reason:
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
type: string
status:
description: status of the condition, one of True, False, Unknown.
enum:
- "True"
- "False"
- Unknown
type: string
type:
description: |-
type of condition in CamelCase or in foo.example.com/CamelCase.
---
Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be
useful (see .node.status.conditions), the ability to deconflict is important.
The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
type: string
required:
- lastTransitionTime
- message
- reason
- status
- type
type: object
type: array
x-kubernetes-list-map-keys:
- type
x-kubernetes-list-type: map
required:
- conditions
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: null
storedVersions: null
---
# Source: calico/templates/kdd-crds.yaml
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
api-approved.kubernetes.io: https://github.com/kubernetes-sigs/network-policy-api/pull/30
policy.networking.k8s.io/bundle-version: v0.1.1
policy.networking.k8s.io/channel: experimental
creationTimestamp: null
name: baselineadminnetworkpolicies.policy.networking.k8s.io
spec:
group: policy.networking.k8s.io
names:
kind: BaselineAdminNetworkPolicy
listKind: BaselineAdminNetworkPolicyList
plural: baselineadminnetworkpolicies
shortNames:
- banp
singular: baselineadminnetworkpolicy
scope: Cluster
versions:
- additionalPrinterColumns:
- jsonPath: .metadata.creationTimestamp
name: Age
type: date
name: v1alpha1
schema:
openAPIV3Schema:
description: |-
BaselineAdminNetworkPolicy is a cluster level resource that is part of the
AdminNetworkPolicy API.
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: Specification of the desired behavior of BaselineAdminNetworkPolicy.
properties:
egress:
description: |-
Egress is the list of Egress rules to be applied to the selected pods if
they are not matched by any AdminNetworkPolicy or NetworkPolicy rules.
A total of 100 Egress rules will be allowed in each BANP instance.
The relative precedence of egress rules within a single BANP object
will be determined by the order in which the rule is written.
Thus, a rule that appears at the top of the egress rules
would take the highest precedence.
BANPs with no egress rules do not affect egress traffic.
Support: Core
items:
description: |-
BaselineAdminNetworkPolicyEgressRule describes an action to take on a particular
set of traffic originating from pods selected by a BaselineAdminNetworkPolicy's
Subject field.
properties:
action:
description: |-
Action specifies the effect this rule will have on matching traffic.
Currently the following actions are supported:
Allow: allows the selected traffic
Deny: denies the selected traffic
Support: Core
enum:
- Allow
- Deny
type: string
name:
description: |-
Name is an identifier for this rule, that may be no more than 100 characters
in length. This field should be used by the implementation to help
improve observability, readability and error-reporting for any applied
BaselineAdminNetworkPolicies.
Support: Core
maxLength: 100
type: string
ports:
description: |-
Ports allows for matching traffic based on port and protocols.
This field is a list of destination ports for the outgoing egress traffic.
If Ports is not set then the rule does not filter traffic via port.
items:
description: |-
AdminNetworkPolicyPort describes how to select network ports on pod(s).
Exactly one field must be set.
maxProperties: 1
minProperties: 1
properties:
namedPort:
description: |-
NamedPort selects a port on a pod(s) based on name.
Support: Extended
type: string
portNumber:
description: |-
Port selects a port on a pod(s) based on number.
Support: Core
properties:
port:
description: |-
Number defines a network port value.
Support: Core
format: int32
maximum: 65535
minimum: 1
type: integer
protocol:
default: TCP
description: |-
Protocol is the network protocol (TCP, UDP, or SCTP) which traffic must
match. If not specified, this field defaults to TCP.
Support: Core
type: string
required:
- port
- protocol
type: object
portRange:
description: |-
PortRange selects a port range on a pod(s) based on provided start and end
values.
Support: Core
properties:
end:
description: |-
End defines a network port that is the end of a port range, the End value
must be greater than Start.
Support: Core
format: int32
maximum: 65535
minimum: 1
type: integer
protocol:
default: TCP
description: |-
Protocol is the network protocol (TCP, UDP, or SCTP) which traffic must
match. If not specified, this field defaults to TCP.
Support: Core
type: string
start:
description: |-
Start defines a network port that is the start of a port range, the Start
value must be less than End.
Support: Core
format: int32
maximum: 65535
minimum: 1
type: integer
required:
- end
- start
type: object
type: object
maxItems: 100
type: array
to:
description: |-
To is the list of destinations whose traffic this rule applies to.
If any AdminNetworkPolicyEgressPeer matches the destination of outgoing
traffic then the specified action is applied.
This field must be defined and contain at least one item.
Support: Core
items:
description: |-
AdminNetworkPolicyEgressPeer defines a peer to allow traffic to.
Exactly one of the selector pointers must be set for a given peer. If a
consumer observes none of its fields are set, they must assume an unknown
option has been specified and fail closed.
maxProperties: 1
minProperties: 1
properties:
namespaces:
description: |-
Namespaces defines a way to select all pods within a set of Namespaces.
Note that host-networked pods are not included in this type of peer.
Support: Core
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
networks:
description: |-
Networks defines a way to select peers via CIDR blocks.
This is intended for representing entities that live outside the cluster,
which can't be selected by pods, namespaces and nodes peers, but note
that cluster-internal traffic will be checked against the rule as
well. So if you Allow or Deny traffic to `"0.0.0.0/0"`, that will allow
or deny all IPv4 pod-to-pod traffic as well. If you don't want that,
add a rule that Passes all pod traffic before the Networks rule.
Each item in Networks should be provided in the CIDR format and should be
IPv4 or IPv6, for example "10.0.0.0/8" or "fd00::/8".
Networks can have upto 25 CIDRs specified.
Support: Extended
items:
description: |-
CIDR is an IP address range in CIDR notation (for example, "10.0.0.0/8" or "fd00::/8").
This string must be validated by implementations using net.ParseCIDR
TODO: Introduce CEL CIDR validation regex isCIDR() in Kube 1.31 when it is available.
maxLength: 43
type: string
x-kubernetes-validations:
- message: CIDR must be either an IPv4 or IPv6 address.
IPv4 address embedded in IPv6 addresses are not
supported
rule: self.contains(':') != self.contains('.')
maxItems: 25
minItems: 1
type: array
x-kubernetes-list-type: set
nodes:
description: |-
Nodes defines a way to select a set of nodes in
the cluster. This field follows standard label selector
semantics; if present but empty, it selects all Nodes.
Support: Extended
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
pods:
description: |-
Pods defines a way to select a set of pods in
a set of namespaces. Note that host-networked pods
are not included in this type of peer.
Support: Core
properties:
namespaceSelector:
description: |-
NamespaceSelector follows standard label selector semantics; if empty,
it selects all Namespaces.
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the
selector applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
podSelector:
description: |-
PodSelector is used to explicitly select pods within a namespace; if empty,
it selects all Pods.
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the
selector applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
required:
- namespaceSelector
- podSelector
type: object
type: object
maxItems: 100
minItems: 1
type: array
required:
- action
- to
type: object
x-kubernetes-validations:
- message: networks/nodes peer cannot be set with namedPorts since
there are no namedPorts for networks/nodes
rule: '!(self.to.exists(peer, has(peer.networks) || has(peer.nodes))
&& has(self.ports) && self.ports.exists(port, has(port.namedPort)))'
maxItems: 100
type: array
ingress:
description: |-
Ingress is the list of Ingress rules to be applied to the selected pods
if they are not matched by any AdminNetworkPolicy or NetworkPolicy rules.
A total of 100 Ingress rules will be allowed in each BANP instance.
The relative precedence of ingress rules within a single BANP object
will be determined by the order in which the rule is written.
Thus, a rule that appears at the top of the ingress rules
would take the highest precedence.
BANPs with no ingress rules do not affect ingress traffic.
Support: Core
items:
description: |-
BaselineAdminNetworkPolicyIngressRule describes an action to take on a particular
set of traffic destined for pods selected by a BaselineAdminNetworkPolicy's
Subject field.
properties:
action:
description: |-
Action specifies the effect this rule will have on matching traffic.
Currently the following actions are supported:
Allow: allows the selected traffic
Deny: denies the selected traffic
Support: Core
enum:
- Allow
- Deny
type: string
from:
description: |-
From is the list of sources whose traffic this rule applies to.
If any AdminNetworkPolicyIngressPeer matches the source of incoming
traffic then the specified action is applied.
This field must be defined and contain at least one item.
Support: Core
items:
description: |-
AdminNetworkPolicyIngressPeer defines an in-cluster peer to allow traffic from.
Exactly one of the selector pointers must be set for a given peer. If a
consumer observes none of its fields are set, they must assume an unknown
option has been specified and fail closed.
maxProperties: 1
minProperties: 1
properties:
namespaces:
description: |-
Namespaces defines a way to select all pods within a set of Namespaces.
Note that host-networked pods are not included in this type of peer.
Support: Core
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
pods:
description: |-
Pods defines a way to select a set of pods in
a set of namespaces. Note that host-networked pods
are not included in this type of peer.
Support: Core
properties:
namespaceSelector:
description: |-
NamespaceSelector follows standard label selector semantics; if empty,
it selects all Namespaces.
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the
selector applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
podSelector:
description: |-
PodSelector is used to explicitly select pods within a namespace; if empty,
it selects all Pods.
properties:
matchExpressions:
description: matchExpressions is a list of label
selector requirements. The requirements are
ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the
selector applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
required:
- namespaceSelector
- podSelector
type: object
type: object
maxItems: 100
minItems: 1
type: array
name:
description: |-
Name is an identifier for this rule, that may be no more than 100 characters
in length. This field should be used by the implementation to help
improve observability, readability and error-reporting for any applied
BaselineAdminNetworkPolicies.
Support: Core
maxLength: 100
type: string
ports:
description: |-
Ports allows for matching traffic based on port and protocols.
This field is a list of ports which should be matched on
the pods selected for this policy i.e the subject of the policy.
So it matches on the destination port for the ingress traffic.
If Ports is not set then the rule does not filter traffic via port.
Support: Core
items:
description: |-
AdminNetworkPolicyPort describes how to select network ports on pod(s).
Exactly one field must be set.
maxProperties: 1
minProperties: 1
properties:
namedPort:
description: |-
NamedPort selects a port on a pod(s) based on name.
Support: Extended
type: string
portNumber:
description: |-
Port selects a port on a pod(s) based on number.
Support: Core
properties:
port:
description: |-
Number defines a network port value.
Support: Core
format: int32
maximum: 65535
minimum: 1
type: integer
protocol:
default: TCP
description: |-
Protocol is the network protocol (TCP, UDP, or SCTP) which traffic must
match. If not specified, this field defaults to TCP.
Support: Core
type: string
required:
- port
- protocol
type: object
portRange:
description: |-
PortRange selects a port range on a pod(s) based on provided start and end
values.
Support: Core
properties:
end:
description: |-
End defines a network port that is the end of a port range, the End value
must be greater than Start.
Support: Core
format: int32
maximum: 65535
minimum: 1
type: integer
protocol:
default: TCP
description: |-
Protocol is the network protocol (TCP, UDP, or SCTP) which traffic must
match. If not specified, this field defaults to TCP.
Support: Core
type: string
start:
description: |-
Start defines a network port that is the start of a port range, the Start
value must be less than End.
Support: Core
format: int32
maximum: 65535
minimum: 1
type: integer
required:
- end
- start
type: object
type: object
maxItems: 100
type: array
required:
- action
- from
type: object
maxItems: 100
type: array
subject:
description: |-
Subject defines the pods to which this BaselineAdminNetworkPolicy applies.
Note that host-networked pods are not included in subject selection.
Support: Core
maxProperties: 1
minProperties: 1
properties:
namespaces:
description: Namespaces is used to select pods via namespace selectors.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
pods:
description: Pods is used to select pods via namespace AND pod
selectors.
properties:
namespaceSelector:
description: |-
NamespaceSelector follows standard label selector semantics; if empty,
it selects all Namespaces.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
podSelector:
description: |-
PodSelector is used to explicitly select pods within a namespace; if empty,
it selects all Pods.
properties:
matchExpressions:
description: matchExpressions is a list of label selector
requirements. The requirements are ANDed.
items:
description: |-
A label selector requirement is a selector that contains values, a key, and an operator that
relates the key and values.
properties:
key:
description: key is the label key that the selector
applies to.
type: string
operator:
description: |-
operator represents a key's relationship to a set of values.
Valid operators are In, NotIn, Exists and DoesNotExist.
type: string
values:
description: |-
values is an array of string values. If the operator is In or NotIn,
the values array must be non-empty. If the operator is Exists or DoesNotExist,
the values array must be empty. This array is replaced during a strategic
merge patch.
items:
type: string
type: array
required:
- key
- operator
type: object
type: array
matchLabels:
additionalProperties:
type: string
description: |-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
map is equivalent to an element of matchExpressions, whose key field is "key", the
operator is "In", and the values array contains only "value". The requirements are ANDed.
type: object
type: object
x-kubernetes-map-type: atomic
required:
- namespaceSelector
- podSelector
type: object
type: object
required:
- subject
type: object
status:
description: Status is the status to be reported by the implementation.
properties:
conditions:
items:
description: "Condition contains details for one aspect of the current
state of this API Resource.\n---\nThis struct is intended for
direct use as an array at the field path .status.conditions. For
example,\n\n\n\ttype FooStatus struct{\n\t // Represents the
observations of a foo's current state.\n\t // Known .status.conditions.type
are: \"Available\", \"Progressing\", and \"Degraded\"\n\t //
+patchMergeKey=type\n\t // +patchStrategy=merge\n\t // +listType=map\n\t
\ // +listMapKey=type\n\t Conditions []metav1.Condition `json:\"conditions,omitempty\"
patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"`\n\n\n\t
\ // other fields\n\t}"
properties:
lastTransitionTime:
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format: date-time
type: string
message:
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength: 32768
type: string
observedGeneration:
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format: int64
minimum: 0
type: integer
reason:
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
type: string
status:
description: status of the condition, one of True, False, Unknown.
enum:
- "True"
- "False"
- Unknown
type: string
type:
description: |-
type of condition in CamelCase or in foo.example.com/CamelCase.
---
Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be
useful (see .node.status.conditions), the ability to deconflict is important.
The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
type: string
required:
- lastTransitionTime
- message
- reason
- status
- type
type: object
type: array
x-kubernetes-list-map-keys:
- type
x-kubernetes-list-type: map
required:
- conditions
type: object
required:
- metadata
- spec
type: object
x-kubernetes-validations:
- message: Only one baseline admin network policy with metadata.name="default"
can be created in the cluster
rule: self.metadata.name == 'default'
served: true
storage: true
subresources:
status: {}
status:
acceptedNames:
kind: ""
plural: ""
conditions: null
storedVersions: null
---
# Source: calico/templates/calico-kube-controllers-rbac.yaml
# Include a clusterrole for the kube-controllers component,
# and bind it to the calico-kube-controllers serviceaccount.
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: calico-kube-controllers
rules:
# Nodes are watched to monitor for deletions.
- apiGroups: [""]
resources:
- nodes
verbs:
- watch
- list
- get
# Pods are watched to check for existence as part of IPAM controller.
- apiGroups: [""]
resources:
- pods
verbs:
- get
- list
- watch
# Services are monitored for service LoadBalancer IP allocation
- apiGroups: [""]
resources:
- services
- services/status
verbs:
- get
- list
- update
- watch
# IPAM resources are manipulated in response to node and block updates, as well as periodic triggers.
- apiGroups: ["crd.projectcalico.org"]
resources:
- ipreservations
verbs:
- list
- apiGroups: ["crd.projectcalico.org"]
resources:
- blockaffinities
- ipamblocks
- ipamhandles
- ipamconfigs
- tiers
verbs:
- get
- list
- create
- update
- delete
- watch
# Pools are watched to maintain a mapping of blocks to IP pools.
- apiGroups: ["crd.projectcalico.org"]
resources:
- ippools
verbs:
- list
- watch
# kube-controllers manages hostendpoints.
- apiGroups: ["crd.projectcalico.org"]
resources:
- hostendpoints
verbs:
- get
- list
- create
- update
- delete
- watch
# Needs access to update clusterinformations.
- apiGroups: ["crd.projectcalico.org"]
resources:
- clusterinformations
verbs:
- get
- list
- create
- update
- watch
# KubeControllersConfiguration is where it gets its config
- apiGroups: ["crd.projectcalico.org"]
resources:
- kubecontrollersconfigurations
verbs:
# read its own config
- get
- list
# create a default if none exists
- create
# update status
- update
# watch for changes
- watch
---
# Source: calico/templates/calico-node-rbac.yaml
# Include a clusterrole for the calico-node DaemonSet,
# and bind it to the calico-node serviceaccount.
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: calico-node
rules:
# Used for creating service account tokens to be used by the CNI plugin
- apiGroups: [""]
resources:
- serviceaccounts/token
resourceNames:
- canal
- calico-cni-plugin
verbs:
- create
# The CNI plugin needs to get pods, nodes, and namespaces.
- apiGroups: [""]
resources:
- pods
- nodes
- namespaces
verbs:
- get
# EndpointSlices are used for Service-based network policy rule
# enforcement.
- apiGroups: ["discovery.k8s.io"]
resources:
- endpointslices
verbs:
- watch
- list
- apiGroups: [""]
resources:
- endpoints
- services
verbs:
# Used to discover service IPs for advertisement.
- watch
- list
# Used to discover Typhas.
- get
# Pod CIDR auto-detection on kubeadm needs access to config maps.
- apiGroups: [""]
resources:
- configmaps
verbs:
- get
- apiGroups: [""]
resources:
- nodes/status
verbs:
# Needed for clearing NodeNetworkUnavailable flag.
- patch
# Calico stores some configuration information in node annotations.
- update
# Watch for changes to Kubernetes NetworkPolicies.
- apiGroups: ["networking.k8s.io"]
resources:
- networkpolicies
verbs:
- watch
- list
# Watch for changes to Kubernetes (Baseline)AdminNetworkPolicies.
- apiGroups: ["policy.networking.k8s.io"]
resources:
- adminnetworkpolicies
- baselineadminnetworkpolicies
verbs:
- watch
- list
# Used by Calico for policy information.
- apiGroups: [""]
resources:
- pods
- namespaces
- serviceaccounts
verbs:
- list
- watch
# The CNI plugin patches pods/status.
- apiGroups: [""]
resources:
- pods/status
verbs:
- patch
# Calico monitors various CRDs for config.
- apiGroups: ["crd.projectcalico.org"]
resources:
- globalfelixconfigs
- felixconfigurations
- bgppeers
- bgpfilters
- globalbgpconfigs
- bgpconfigurations
- ippools
- ipreservations
- ipamblocks
- globalnetworkpolicies
- stagedglobalnetworkpolicies
- networkpolicies
- stagednetworkpolicies
- stagedkubernetesnetworkpolicies
- globalnetworksets
- networksets
- clusterinformations
- hostendpoints
- blockaffinities
- caliconodestatuses
- tiers
verbs:
- get
- list
- watch
# Calico creates some tiers on startup.
- apiGroups: ["crd.projectcalico.org"]
resources:
- tiers
verbs:
- create
# Calico must create and update some CRDs on startup.
- apiGroups: ["crd.projectcalico.org"]
resources:
- ippools
- felixconfigurations
- clusterinformations
verbs:
- create
- update
# Calico must update some CRDs.
- apiGroups: ["crd.projectcalico.org"]
resources:
- caliconodestatuses
verbs:
- update
# Calico stores some configuration information on the node.
- apiGroups: [""]
resources:
- nodes
verbs:
- get
- list
- watch
# These permissions are only required for upgrade from v2.6, and can
# be removed after upgrade or on fresh installations.
- apiGroups: ["crd.projectcalico.org"]
resources:
- bgpconfigurations
- bgppeers
verbs:
- create
- update
---
# Source: calico/templates/calico-node-rbac.yaml
# CNI cluster role
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: calico-cni-plugin
rules:
- apiGroups: [""]
resources:
- pods
- nodes
- namespaces
verbs:
- get
- apiGroups: [""]
resources:
- pods/status
verbs:
- patch
---
# Source: calico/templates/calico-node-rbac.yaml
# Flannel ClusterRole
# Pulled from https://github.com/flannel-io/flannel/blob/master/Documentation/k8s-old-manifests/kube-flannel-rbac.yml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: flannel
rules:
- apiGroups: [""]
resources:
- pods
verbs:
- get
- apiGroups: [""]
resources:
- nodes
verbs:
- list
- watch
- apiGroups: [""]
resources:
- nodes/status
verbs:
- patch
---
# Source: calico/templates/tier-getter.yaml
# Implements the necessary permissions for the kube-controller-manager to interact with
# Tiers and Tiered Policies for GC.
#
# https://github.com/tigera/operator/blob/v1.37.0/pkg/render/apiserver.go#L1505-L1545
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: calico-tier-getter
rules:
- apiGroups:
- "projectcalico.org"
resources:
- "tiers"
verbs:
- "get"
---
# Source: calico/templates/calico-kube-controllers-rbac.yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: calico-kube-controllers
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: calico-kube-controllers
subjects:
- kind: ServiceAccount
name: calico-kube-controllers
namespace: kube-system
---
# Source: calico/templates/calico-node-rbac.yaml
# Bind the flannel ClusterRole to the canal ServiceAccount.
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: canal-flannel
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: flannel
subjects:
- kind: ServiceAccount
name: canal
namespace: kube-system
---
# Source: calico/templates/calico-node-rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: canal-calico
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: calico-node
subjects:
- kind: ServiceAccount
name: canal
namespace: kube-system
---
# Source: calico/templates/calico-node-rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: calico-cni-plugin
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: calico-cni-plugin
subjects:
- kind: ServiceAccount
name: calico-cni-plugin
namespace: kube-system
---
# Source: calico/templates/tier-getter.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: calico-tier-getter
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: calico-tier-getter
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: system:kube-controller-manager
---
# Source: calico/templates/calico-node.yaml
# This manifest installs the canal container, as well
# as the CNI plugins and network config on
# each master and worker node in a Kubernetes cluster.
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: canal
namespace: kube-system
labels:
k8s-app: canal
spec:
selector:
matchLabels:
k8s-app: canal
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
template:
metadata:
labels:
k8s-app: canal
spec:
nodeSelector:
kubernetes.io/os: linux
hostNetwork: true
tolerations:
# Make sure canal gets scheduled on all nodes.
- effect: NoSchedule
operator: Exists
# Mark the pod as a critical add-on for rescheduling.
- key: CriticalAddonsOnly
operator: Exists
- effect: NoExecute
operator: Exists
serviceAccountName: canal
securityContext:
seccompProfile:
type: RuntimeDefault
# Minimize downtime during a rolling upgrade or deletion; tell Kubernetes to do a "force
# deletion": https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods.
terminationGracePeriodSeconds: 0
priorityClassName: system-node-critical
initContainers:
# This container installs the CNI binaries
# and CNI network config file on each node.
- name: install-cni
image: docker.io/calico/cni:v3.30.0
imagePullPolicy: IfNotPresent
command: ["/opt/cni/bin/install"]
envFrom:
- configMapRef:
# Allow KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT to be overridden for eBPF mode.
name: kubernetes-services-endpoint
optional: true
env:
# Set the serviceaccount name to use for the Calico CNI plugin.
# We use canal-node instead of calico-node when using flannel networking.
- name: CALICO_CNI_SERVICE_ACCOUNT
valueFrom:
fieldRef:
fieldPath: spec.serviceAccountName
# Name of the CNI config file to create.
- name: CNI_CONF_NAME
value: "10-canal.conflist"
# The CNI network config to install on each node.
- name: CNI_NETWORK_CONFIG
valueFrom:
configMapKeyRef:
name: canal-config
key: cni_network_config
# Set the hostname based on the k8s node name.
- name: KUBERNETES_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
# CNI MTU Config variable
- name: CNI_MTU
valueFrom:
configMapKeyRef:
name: canal-config
key: veth_mtu
# Prevents the container from sleeping forever.
- name: SLEEP
value: "false"
volumeMounts:
- mountPath: /host/opt/cni/bin
name: cni-bin-dir
- mountPath: /host/etc/cni/net.d
name: cni-net-dir
securityContext:
privileged: true
# This init container mounts the necessary filesystems needed by the BPF data plane
# i.e. bpf at /sys/fs/bpf and cgroup2 at /run/calico/cgroup. Calico-node initialisation is executed
# in best effort fashion, i.e. no failure for errors, to not disrupt pod creation in iptable mode.
# - name: "mount-bpffs"
# image: docker.io/calico/node:v3.30.0
# imagePullPolicy: IfNotPresent
# command: ["calico-node", "-init", "-best-effort"]
# volumeMounts:
# - mountPath: /sys/fs
# name: sys-fs
# # Bidirectional is required to ensure that the new mount we make at /sys/fs/bpf propagates to the host
# # so that it outlives the init container.
# mountPropagation: Bidirectional
# - mountPath: /var/run/calico
# name: var-run-calico
# # Bidirectional is required to ensure that the new mount we make at /run/calico/cgroup propagates to the host
# # so that it outlives the init container.
# mountPropagation: Bidirectional
# # Mount /proc/ from host which usually is an init program at /nodeproc. It's needed by mountns binary,
# # executed by calico-node, to mount root cgroup2 fs at /run/calico/cgroup to attach CTLB programs correctly.
# - mountPath: /nodeproc
# name: nodeproc
# readOnly: true
# securityContext:
# privileged: true
containers:
# Runs canal container on each Kubernetes node. This
# container programs network policy and routes on each
# host.
- name: calico-node
image: docker.io/calico/node:v3.30.0
imagePullPolicy: IfNotPresent
envFrom:
- configMapRef:
# Allow KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT to be overridden for eBPF mode.
name: kubernetes-services-endpoint
optional: true
env:
# Use Kubernetes API as the backing datastore.
- name: DATASTORE_TYPE
value: "kubernetes"
# Configure route aggregation based on pod CIDR.
- name: USE_POD_CIDR
value: "true"
# Wait for the datastore.
- name: WAIT_FOR_DATASTORE
value: "true"
# Set based on the k8s node name.
- name: NODENAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
# Set the serviceaccount name to use for the Calico CNI plugin.
# We use canal-node instead of calico-node when using flannel networking.
- name: CALICO_CNI_SERVICE_ACCOUNT
valueFrom:
fieldRef:
fieldPath: spec.serviceAccountName
# Don't enable BGP.
- name: CALICO_NETWORKING_BACKEND
value: "none"
# Cluster type to identify the deployment type
- name: CLUSTER_TYPE
value: "k8s,canal"
# Period, in seconds, at which felix re-applies all iptables state
- name: FELIX_IPTABLESREFRESHINTERVAL
value: "60"
# No IP address needed.
- name: IP
value: ""
# The default IPv4 pool to create on startup if none exists. Pod IPs will be
# chosen from this range. Changing this value after installation will have
# no effect. This should fall within `--cluster-cidr`.
# - name: CALICO_IPV4POOL_CIDR
# value: "192.168.0.0/16"
# Disable file logging so `kubectl logs` works.
- name: CALICO_DISABLE_FILE_LOGGING
value: "true"
# Set Felix endpoint to host default action to ACCEPT.
- name: FELIX_DEFAULTENDPOINTTOHOSTACTION
value: "ACCEPT"
# Disable IPv6 on Kubernetes.
- name: FELIX_IPV6SUPPORT
value: "false"
- name: FELIX_HEALTHENABLED
value: "true"
securityContext:
privileged: true
resources:
requests:
cpu: 250m
lifecycle:
preStop:
exec:
command:
- /bin/calico-node
- -shutdown
livenessProbe:
exec:
command:
- /bin/calico-node
- -felix-live
periodSeconds: 10
initialDelaySeconds: 10
failureThreshold: 6
timeoutSeconds: 10
readinessProbe:
httpGet:
path: /readiness
port: 9099
host: localhost
periodSeconds: 10
timeoutSeconds: 10
volumeMounts:
# For maintaining CNI plugin API credentials.
- mountPath: /host/etc/cni/net.d
name: cni-net-dir
readOnly: false
- mountPath: /lib/modules
name: lib-modules
readOnly: true
- mountPath: /run/xtables.lock
name: xtables-lock
readOnly: false
- mountPath: /var/run/calico
name: var-run-calico
readOnly: false
- mountPath: /var/lib/calico
name: var-lib-calico
readOnly: false
- name: policysync
mountPath: /var/run/nodeagent
# For eBPF mode, we need to be able to mount the BPF filesystem at /sys/fs/bpf so we mount in the
# parent directory.
# - name: bpffs
# mountPath: /sys/fs/bpf
- name: cni-log-dir
mountPath: /var/log/calico/cni
readOnly: true
# This container runs flannel using the kube-subnet-mgr backend
# for allocating subnets.
- name: kube-flannel
image: docker.io/flannel/flannel:v0.24.4
imagePullPolicy: IfNotPresent
command: [ "/opt/bin/flanneld", "--ip-masq", "--kube-subnet-mgr" ]
securityContext:
privileged: true
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: FLANNELD_IFACE
valueFrom:
configMapKeyRef:
name: canal-config
key: canal_iface
- name: FLANNELD_IP_MASQ
valueFrom:
configMapKeyRef:
name: canal-config
key: masquerade
volumeMounts:
- mountPath: /run/xtables.lock
name: xtables-lock
readOnly: false
- name: flannel-cfg
mountPath: /etc/kube-flannel/
volumes:
# Used by canal.
- name: lib-modules
hostPath:
path: /lib/modules
- name: var-run-calico
hostPath:
path: /var/run/calico
type: DirectoryOrCreate
- name: var-lib-calico
hostPath:
path: /var/lib/calico
type: DirectoryOrCreate
- name: xtables-lock
hostPath:
path: /run/xtables.lock
type: FileOrCreate
# - name: sys-fs
# hostPath:
# path: /sys/fs/
# type: DirectoryOrCreate
# - name: bpffs
# hostPath:
# path: /sys/fs/bpf
# type: Directory
# mount /proc at /nodeproc to be used by mount-bpffs initContainer to mount root cgroup2 fs.
# - name: nodeproc
# hostPath:
# path: /proc
# Used by flannel.
- name: flannel-cfg
configMap:
name: canal-config
# Used to install CNI.
- name: cni-bin-dir
hostPath:
path: /opt/cni/bin
type: DirectoryOrCreate
- name: cni-net-dir
hostPath:
path: /etc/cni/net.d
# Used to access CNI logs.
- name: cni-log-dir
hostPath:
path: /var/log/calico/cni
# Used to create per-pod Unix Domain Sockets
- name: policysync
hostPath:
type: DirectoryOrCreate
path: /var/run/nodeagent
---
# Source: calico/templates/calico-kube-controllers.yaml
# See https://github.com/projectcalico/kube-controllers
apiVersion: apps/v1
kind: Deployment
metadata:
name: calico-kube-controllers
namespace: kube-system
labels:
k8s-app: calico-kube-controllers
spec:
# The controllers can only have a single active instance.
replicas: 1
selector:
matchLabels:
k8s-app: calico-kube-controllers
strategy:
type: Recreate
template:
metadata:
name: calico-kube-controllers
namespace: kube-system
labels:
k8s-app: calico-kube-controllers
spec:
nodeSelector:
kubernetes.io/os: linux
tolerations:
# Mark the pod as a critical add-on for rescheduling.
- key: CriticalAddonsOnly
operator: Exists
- key: node-role.kubernetes.io/master
effect: NoSchedule
- key: node-role.kubernetes.io/control-plane
effect: NoSchedule
serviceAccountName: calico-kube-controllers
securityContext:
seccompProfile:
type: RuntimeDefault
priorityClassName: system-cluster-critical
containers:
- name: calico-kube-controllers
image: docker.io/calico/kube-controllers:v3.30.0
imagePullPolicy: IfNotPresent
env:
# Choose which controllers to run.
- name: ENABLED_CONTROLLERS
value: node,loadbalancer
- name: DATASTORE_TYPE
value: kubernetes
livenessProbe:
exec:
command:
- /usr/bin/check-status
- -l
periodSeconds: 10
initialDelaySeconds: 10
failureThreshold: 6
timeoutSeconds: 10
readinessProbe:
exec:
command:
- /usr/bin/check-status
- -r
periodSeconds: 10
securityContext:
runAsNonRoot: true
================================================
FILE: resources/custom-cni/flannel.yaml
================================================
apiVersion: v1
kind: Namespace
metadata:
labels:
k8s-app: flannel
pod-security.kubernetes.io/enforce: privileged
name: kube-flannel
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: flannel
name: flannel
namespace: kube-flannel
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
k8s-app: flannel
name: flannel
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- patch
- apiGroups:
- networking.k8s.io
resources:
- clustercidrs
verbs:
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
k8s-app: flannel
name: flannel
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: flannel
subjects:
- kind: ServiceAccount
name: flannel
namespace: kube-flannel
---
apiVersion: v1
data:
cni-conf.json: |
{
"name": "cbr0",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
net-conf.json: |
{
"Network": "172.17.0.0/16",
"Backend": {
"Type": "vxlan"
}
}
kind: ConfigMap
metadata:
labels:
app: flannel
k8s-app: flannel
tier: node
name: kube-flannel-cfg
namespace: kube-flannel
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
labels:
app: flannel
k8s-app: flannel
tier: node
name: kube-flannel-ds
namespace: kube-flannel
spec:
selector:
matchLabels:
app: flannel
k8s-app: flannel
template:
metadata:
labels:
app: flannel
k8s-app: flannel
tier: node
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/os
operator: In
values:
- linux
containers:
- args:
- --ip-masq
- --kube-subnet-mgr
- --iface=eth0
command:
- /opt/bin/flanneld
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: EVENT_QUEUE_DEPTH
value: "5000"
image: docker.io/flannel/flannel:v0.23.0
name: kube-flannel
resources:
requests:
cpu: 100m
memory: 50Mi
securityContext:
capabilities:
add:
- NET_ADMIN
- NET_RAW
privileged: false
volumeMounts:
- mountPath: /run/flannel
name: run
- mountPath: /etc/kube-flannel/
name: flannel-cfg
- mountPath: /run/xtables.lock
name: xtables-lock
hostNetwork: true
initContainers:
- args:
- -f
- /flannel
- /opt/cni/bin/flannel
command:
- cp
image: docker.io/flannel/flannel-cni-plugin:v1.2.0
name: install-cni-plugin
volumeMounts:
- mountPath: /opt/cni/bin
name: cni-plugin
- args:
- -f
- /etc/kube-flannel/cni-conf.json
- /etc/cni/net.d/10-flannel.conflist
command:
- cp
image: docker.io/flannel/flannel:v0.23.0
name: install-cni
volumeMounts:
- mountPath: /etc/cni/net.d
name: cni
- mountPath: /etc/kube-flannel/
name: flannel-cfg
priorityClassName: system-node-critical
serviceAccountName: flannel
tolerations:
- effect: NoSchedule
operator: Exists
volumes:
- hostPath:
path: /run/flannel
name: run
- hostPath:
path: /opt/cni/bin
name: cni-plugin
- hostPath:
path: /etc/cni/net.d
name: cni
- configMap:
name: kube-flannel-cfg
name: flannel-cfg
- hostPath:
path: /run/xtables.lock
type: FileOrCreate
name: xtables-lock
================================================
FILE: resources/elastic-app-cka02-arch.yaml
================================================
apiVersion: v1
kind: Pod
metadata:
name: elastic-app-cka02-arch
spec:
containers:
- name: elastic-app
image: busybox:1.28
args:
- /bin/sh
- -c
- >
mkdir /var/log;
i=0;
while true;
do
echo "$(date) INFO $i" >> /var/log/elastic-app.log;
i=$((i+1));
sleep 1;
done
volumeMounts:
- name: varlog
mountPath: /var/log
volumes:
- name: varlog
emptyDir: {}
================================================
FILE: resources/essports-wl02.yaml
================================================
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: essports-wl02
name: essports-wl02
spec:
replicas: 1
selector:
matchLabels:
app: essports-wl02
template:
metadata:
labels:
app: essports-wl02
spec:
containers:
- image: nginx:alpine
name: nginx
---
apiVersion: v1
kind: Service
metadata:
labels:
app: essports-svc
name: essports-svc
spec:
ports:
- name: 80-32767
port: 80
protocol: TCP
targetPort: 80
nodePort: 32750
selector:
app: essports-wl02
type: NodePort
================================================
FILE: resources/frontend-wl04.yaml
================================================
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: frontend-wl04
name: frontend-wl04
spec:
replicas: 2
selector:
matchLabels:
app: frontend-wl04
template:
metadata:
labels:
app: frontend-wl04
spec:
containers:
- image: kodekloud/webapp-color:v1
name: simple-webapp
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
labels:
app: frontend-wl04-svc
name: frontend-wl04-svc
spec:
ports:
- name: 8080-8080
port: 8080
protocol: TCP
targetPort: 8080
nodePort: 30080
selector:
app: frontend-wl04
type: NodePort
================================================
FILE: resources/high_cpu_node.sh
================================================
#!/bin/bash
echo cluster1 $(kubectl --context cluster1 top node --no-headers | sort -nr -k2 | head -1) > /tmp/high_cpu_node
echo cluster2 $(kubectl --context cluster2 top node --no-headers | sort -nr -k2 | head -1) >> /tmp/high_cpu_node
echo cluster3 $(kubectl --context cluster3 top node --no-headers | sort -nr -k2 | head -1) >> /tmp/high_cpu_node
echo cluster4 $(kubectl --context cluster4 top node --no-headers | sort -nr -k2 | head -1) >> /tmp/high_cpu_node
final_value=$(cat /tmp/high_cpu_node | sort -nr -k 3 | awk '{print $1,$2}' | head -1 | tr " " ,)
if [[ $(cat /opt/high_cpu_node | grep $final_value) ]]
then
echo SUCCESS
else
echo FAIL
fi
================================================
FILE: resources/high_cpu_pod.sh
================================================
#!/bin/bash
echo cluster1 $(kubectl --context cluster1 top pods -A --no-headers | sort -nr -k3 | head -1) > /tmp/high_cpu_pod
echo cluster2 $(kubectl --context cluster2 top pods -A --no-headers | sort -nr -k3 | head -1) >> /tmp/high_cpu_pod
echo cluster3 $(kubectl --context cluster3 top pods -A --no-headers | sort -nr -k3 | head -1) >> /tmp/high_cpu_pod
echo cluster4 $(kubectl --context cluster4 top pods -A --no-headers | sort -nr -k3 | head -1) >> /tmp/high_cpu_pod
final_value=$(cat /tmp/high_cpu_pod | sort -nr -k 4 | awk '{print $1,$2,$3}' | head -1 | tr " " ,)
if [[ $(cat /opt/high_cpu_pod | grep $final_value) ]]
then
echo SUCCESS
else
echo FAIL
fi
================================================
FILE: resources/high_cpu_pod.yaml
================================================
apiVersion: v1
kind: Pod
metadata:
name: frontend-stable-cka05-arch
spec:
containers:
- name: fe-cka05-arch
image: vish/stress
resources:
limits:
cpu: "0.3"
requests:
cpu: "0.2"
args:
- -cpus
- "1"
================================================
FILE: resources/high_cpu_pod_1.yaml
================================================
apiVersion: v1
kind: Pod
metadata:
name: frontend-stable-cka05-arch
spec:
nodeName: cluster1-controlplane
containers:
- name: fe-cka05-arch
image: vish/stress
resources:
limits:
cpu: "0.395"
requests:
cpu: "0.35"
args:
- -cpus
- "1"
================================================
FILE: resources/high_memory_node.sh
================================================
#!/bin/bash
echo cluster1 $(kubectl --context cluster1 top node --no-headers | sort -nr -k4 | head -1) > /tmp/high_memory_node
echo cluster2 $(kubectl --context cluster2 top node --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_node
echo cluster3 $(kubectl --context cluster3 top node --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_node
echo cluster4 $(kubectl --context cluster4 top node --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_node
final_value=$(cat /tmp/high_memory_node | sort -nr -k 5 | awk '{print $1,$2}' | head -1 | tr " " ,)
if [[ $(cat /opt/high_memory_node | grep $final_value) ]]
then
echo SUCCESS
else
echo FAIL
fi
================================================
FILE: resources/high_memory_pod.sh
================================================
#!/bin/bash
echo cluster1 $(kubectl --context cluster1 top pods -A --no-headers | sort -nr -k4 | head -1) > /tmp/high_memory_pod
echo cluster2 $(kubectl --context cluster2 top pods -A --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_pod
echo cluster3 $(kubectl --context cluster3 top pods -A --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_pod
echo cluster4 $(kubectl --context cluster4 top pods -A --no-headers | sort -nr -k4 | head -1) >> /tmp/high_memory_pod
final_value=$(cat /tmp/high_memory_pod | sort -nr -k 5 | awk '{print $1,$2,$3}' | head -1 | tr " " ,)
if [[ $(cat /opt/high_memory_pod | grep $final_value) ]]
then
echo SUCCESS
else
echo FAIL
fi
================================================
FILE: resources/high_memory_pod.yaml
================================================
apiVersion: v1
kind: Pod
metadata:
name: backend-cka06-arch
namespace: default
spec:
containers:
- name: memory-demo-ctr
image: polinux/stress
resources:
requests:
memory: "200Mi"
limits:
memory: "700Mi"
command: ["stress"]
args: ["--vm", "1", "--vm-bytes", "600M", "--vm-hang", "1"]
================================================
FILE: resources/high_memory_pod_1.yaml
================================================
apiVersion: v1
kind: Pod
metadata:
name: backend-cka06-arch
namespace: default
spec:
nodeName: cluster1-node02
containers:
- name: memory-demo-ctr
image: polinux/stress
resources:
requests:
memory: "1G"
limits:
memory: "1.2G"
command: ["stress"]
args: ["--vm", "1", "--vm-bytes", "1G", "--vm-hang", "1"]
================================================
FILE: resources/nginx-wl06.yaml
================================================
---
apiVersion: v1
kind: Pod
metadata:
labels:
run: nginx-wl06
name: nginx-wl06
spec:
containers:
- image: nginx
name: nginx-wl06
resources:
requests:
memory: 100Gi
================================================
FILE: resources/ns10-apd.yaml
================================================
---
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: null
name: security-alpha-01
---
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: null
name: digi-locker-02
---
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: null
name: web-dashboard-03
---
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: null
name: atlanta-page-04
================================================
FILE: resources/staging-scripts/alpha-ns-apd-13.yaml
================================================
---
apiVersion: v1
kind: Namespace
metadata:
name: alpha-ns-apd
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ruby-alpha-apd
labels:
mark-zxc: alpha1
namespace: alpha-ns-apd
spec:
replicas: 5
selector:
matchLabels:
alpha: v1
template:
metadata:
labels:
alpha: v1
spec:
containers:
- image: kodekloud/webapp-color:v1
name: ruby-alpha-apd
---
apiVersion: v1
kind: Service
metadata:
labels:
app: alpha-apd-service
name: alpha-apd-service
namespace: alpha-ns-apd
spec:
type: NodePort
ports:
- port: 8080
protocol: TCP
targetPort: 8080
selector:
alpha: v1
status:
loadBalancer: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
mark-zyx: alpha2
name: cube-alpha-apd
namespace: alpha-ns-apd
spec:
replicas: 5
selector:
matchLabels:
alpha: v1
template:
metadata:
labels:
alpha: v1
spec:
containers:
- image: kodekloud/webapp-color:v2
name: cube-alpha-apd
================================================
FILE: resources/staging-scripts/check-connection.sh
================================================
#!/bin/bash
if [[ $(ssh $1 kubectl exec $2 -- nc -w 2 $3 $4) ]];
then
echo success;
else
echo fail;
fi
================================================
FILE: resources/staging-scripts/circle-apd15.yaml
================================================
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
tier-i: circle-apd
app-kgh: route01
name: circle-apd
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app-kgh: route01
template:
metadata:
labels:
app-kgh: route01
spec:
containers:
- image: kodekloud/webapp-color:v1
name: circle-apd
---
apiVersion: v1
kind: Service
metadata:
labels:
app: foundary-svc
name: foundary-svc
namespace: default
spec:
type: NodePort
ports:
- port: 8080
protocol: TCP
targetPort: 8080
selector:
app-kgh: route01
status:
loadBalancer: {}
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
tier-ii: square-apd
app-xnz: route02
name: square-apd
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app-xnz: route02
template:
metadata:
labels:
app-xnz: route02
spec:
containers:
- image: kodekloud/webapp-color:v2
name: square-apd
================================================
FILE: resources/staging-scripts/create_user_certs.sh
================================================
#!/bin/bash
create_user_cert(){
USER=$1
CERTS_PATH=/etc/kubernetes/pki/users/${USER}
echo "Creating certificates for user ${USER}"
mkdir -p ${CERTS_PATH}
openssl genrsa -out ${CERTS_PATH}/${USER}.key 2048
openssl req -new -key ${CERTS_PATH}/${USER}.key -subj "/CN=${USER}" -out ${CERTS_PATH}/${USER}.csr
openssl x509 -req -in ${CERTS_PATH}/${USER}.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out ${CERTS_PATH}/${USER}.crt
}
if [[ $# -eq 0 ]] ; then
echo "No arguments supplied. Username(s) must be supplied. If multiple separate by space."
exit 1
fi
for user in "$@"
do
create_user_cert $user
done
================================================
FILE: resources/staging-scripts/deploy.yaml
================================================
#GENERATED FOR K8S 1.20
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- endpoints
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resourceNames:
- ingress-controller-leader
resources:
- configmaps
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-admission
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
- namespaces
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-admission
rules:
- apiGroups:
- admissionregistration.k8s.io
resources:
- validatingwebhookconfigurations
verbs:
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-admission
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: v1
data:
allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-controller
namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
externalTrafficPolicy: Local
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
nodePort: 30080
- appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
nodePort: 32103
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: NodePort
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
ports:
- appProtocol: https
name: https-webhook
port: 443
targetPort: webhook
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: ClusterIP
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
spec:
containers:
- args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
- --election-id=ingress-controller-leader
- --watch-ingress-without-class=true
- --default-backend-service=app-space/default-backend-service
- --controller-class=k8s.io/ingress-nginx
- --ingress-class=nginx
- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: LD_PRELOAD
value: /usr/local/lib/libmimalloc.so
image: k8s.gcr.io/ingress-nginx/controller:v1.1.2@sha256:28b11ce69e57843de44e3db6413e98d09de0f6688e33d4bd384002a44f78405c
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
livenessProbe:
failureThreshold: 5
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: controller
ports:
- containerPort: 80
name: http
protocol: TCP
- containerPort: 443
name: https
protocol: TCP
- containerPort: 8443
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 90Mi
securityContext:
allowPrivilegeEscalation: true
capabilities:
add:
- NET_BIND_SERVICE
drop:
- ALL
runAsUser: 101
volumeMounts:
- mountPath: /usr/local/certificates/
name: webhook-cert
readOnly: true
dnsPolicy: ClusterFirst
nodeSelector:
kubernetes.io/os: linux
serviceAccountName: ingress-nginx
terminationGracePeriodSeconds: 300
volumes:
- name: webhook-cert
secret:
secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
annotations:
helm.sh/hook: pre-install,pre-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-admission-create
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-admission-create
spec:
containers:
- args:
- create
- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
- --namespace=$(POD_NAMESPACE)
- --secret-name=ingress-nginx-admission
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
imagePullPolicy: IfNotPresent
name: create
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
annotations:
helm.sh/hook: post-install,post-upgrade
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-admission-patch
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-admission-patch
spec:
containers:
- args:
- patch
- --webhook-name=ingress-nginx-admission
- --namespace=$(POD_NAMESPACE)
- --patch-mutating=false
- --secret-name=ingress-nginx-admission
- --patch-failure-policy=Fail
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
imagePullPolicy: IfNotPresent
name: patch
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: nginx
spec:
controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.1.2
helm.sh/chart: ingress-nginx-4.0.18
name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
service:
name: ingress-nginx-controller-admission
namespace: ingress-nginx
path: /networking/v1/ingresses
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.nginx.ingress.kubernetes.io
rules:
- apiGroups:
- networking.k8s.io
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- ingresses
sideEffects: None
================================================
FILE: resources/staging-scripts/foundary-apd6.yaml
================================================
---
apiVersion: v1
kind: Namespace
metadata:
name: blue-apd
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: foundary-apd
name: foundary-apd
namespace: blue-apd
spec:
replicas: 1
selector:
matchLabels:
app: foundary-apd
template:
metadata:
labels:
app: foundary-apd
spec:
containers:
- command:
- sh
- -c
- "sleep 9500"
image: ubuntu:jammy
name: foundary-apd
================================================
FILE: resources/staging-scripts/get-highest-pod.sh
================================================
#!/bin/bash
get_highest_pod() {
local highest=0
local highest_pod=""
local highest_cluster=""
local metric_key=""
if [[ "$1" == "cpu" ]]; then
metric_key="3"
elif [[ "$1" == "memory" ]]; then
metric_key="4"
else
echo "Invalid argument. Usage: $0 [cpu|memory]"
return 1
fi
for cluster in cluster1 cluster2 cluster3 cluster4;
do
pod_info=$(kubectl top pods -A --context $cluster --no-headers | sort -nr -k${metric_key} | head -1 | awk '{print $1","$2","$'$metric_key'}')
current_metric=$(echo $pod_info | awk -F',' '{print $3}' | sed 's/m//' | sed 's/Mi//')
if (( $(echo "$current_metric $highest" | awk '{if ($1 > $2) print 1; else print 0}') )); then
highest=$current_metric
highest_pod=$pod_info
highest_cluster=$cluster
fi
done
highest_pod=${highest_pod%,*} # Remove everything after the last comma
echo $highest_cluster,$highest_pod
}
get_highest_pod $@
================================================
FILE: resources/staging-scripts/ingress-application-ckad.yaml
================================================
---
kind: Namespace
apiVersion: v1
metadata:
name: app-space
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp-wear
namespace: app-space
spec:
replicas: 1
selector:
matchLabels:
app: webapp-wear
template:
metadata:
labels:
app: webapp-wear
spec:
containers:
- name: simple-webapp
image: kodekloud/ecommerce:apparels
imagePullPolicy: Always
ports:
- containerPort: 8080
---
kind: Service
apiVersion: v1
metadata:
name: wear-service
namespace: app-space
spec:
selector:
app: webapp-wear
ports:
- port: 8080
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp-video
namespace: app-space
spec:
replicas: 1
selector:
matchLabels:
app: webapp-video
template:
metadata:
labels:
app: webapp-video
spec:
containers:
- name: simple-webapp
image: kodekloud/ecommerce:video
imagePullPolicy: Always
ports:
- containerPort: 8080
---
kind: Service
apiVersion: v1
metadata:
name: video-service
namespace: app-space
spec:
selector:
app: webapp-video
ports:
- port: 8080
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: default-backend
namespace: app-space
spec:
replicas: 1
selector:
matchLabels:
app: default-backend
template:
metadata:
labels:
app: default-backend
spec:
containers:
- name: simple-webapp
imagePullPolicy: Always
image: kodekloud/ecommerce:404
ports:
- containerPort: 8080
---
kind: Service
apiVersion: v1
metadata:
name: default-backend-service
namespace: app-space
spec:
selector:
app: default-backend
ports:
- port: 80
targetPort: 8080
================================================
FILE: resources/staging-scripts/ingress-controller-ckad.yaml
================================================
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- apiGroups:
- ""
resources:
- configmaps
- pods
- secrets
- endpoints
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resourceNames:
- ingress-controller-leader
resources:
- configmaps
verbs:
- get
- update
- apiGroups:
- ""
resources:
- configmaps
verbs:
- create
- apiGroups:
- coordination.k8s.io
resourceNames:
- ingress-controller-leader
resources:
- leases
verbs:
- get
- update
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- create
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
namespace: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
rules:
- apiGroups:
- ""
resources:
- configmaps
- endpoints
- nodes
- pods
- secrets
- namespaces
verbs:
- list
- watch
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list
- watch
- apiGroups:
- networking.k8s.io
resources:
- ingresses
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- apiGroups:
- networking.k8s.io
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- networking.k8s.io
resources:
- ingressclasses
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
rules:
- apiGroups:
- admissionregistration.k8s.io
resources:
- validatingwebhookconfigurations
verbs:
- get
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
- kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
---
apiVersion: v1
data:
allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
externalTrafficPolicy: Local
ports:
- appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
- appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
ports:
- appProtocol: https
name: https-webhook
port: 443
targetPort: webhook
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: ClusterIP
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-create
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-create
spec:
containers:
- args:
- create
- --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
- --namespace=$(POD_NAMESPACE)
- --secret-name=ingress-nginx-admission
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
imagePullPolicy: IfNotPresent
name: create
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-patch
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-patch
spec:
containers:
- args:
- patch
- --webhook-name=ingress-nginx-admission
- --namespace=$(POD_NAMESPACE)
- --patch-mutating=false
- --secret-name=ingress-nginx-admission
- --patch-failure-policy=Fail
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
imagePullPolicy: IfNotPresent
name: patch
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: nginx
spec:
controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
service:
name: ingress-nginx-controller-admission
namespace: ingress-nginx
path: /networking/v1/ingresses
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.nginx.ingress.kubernetes.io
rules:
- apiGroups:
- networking.k8s.io
apiVersions:
- v1
operations:
- CREATE
- UPDATE
resources:
- ingresses
sideEffects: None
================================================
FILE: resources/staging-scripts/ingress-resource-ckad.yaml
================================================
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-resource-svcn
namespace: app-space
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
ingressClassName: nginx
rules:
- http:
paths:
- path: /wear
pathType: Prefix
backend:
service:
name: wear-service
port:
number: 8080
- path: /watch
pathType: Prefix
backend:
service:
name: video-service
port:
number: 8080
================================================
FILE: resources/staging-scripts/ingress-staging-cka04-svcn.sh
================================================
#!/bin/bash
export KUBECONFIG=/root/.kube/clusters/cluster3_config
cat < /tmp/index.html
Hello World!
Hello World!
EOF
kubectl create configmap test-html --from-file /tmp/index.html
kubectl apply -f - < /lib/systemd/system/webserver.service
[Unit]
Description=kubectl proxy 8888
After=network.target
[Service]
User=root
ExecStart=/bin/bash -c "/usr/bin/kubectl --kubeconfig=/root/.kube/clusters/cluster2_config port-forward --address 0.0.0.0 -n kube-public server1-cka03-svcn 9999:80"
StartLimitInterval=0
RestartSec=10
Restart=always
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable webserver.service --now
kubectl --context=cluster3 apply -f - << EOF
apiVersion: v1
kind: Service
metadata:
name: external-webserver-cka03-svcn
spec:
ports:
- protocol: TCP
port: 80
targetPort: 9999
EOF
================================================
FILE: resources/staging-scripts/news-apd.yaml
================================================
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: news-apd
spec:
selector:
matchLabels:
name: news-apd
replicas: 1
template:
metadata:
labels:
name: news-apd
spec:
containers:
- name: news-apd-container
image: busybox:1.28
command:
- sh
- -c
- "sleeeep 45000"
================================================
FILE: resources/staging-scripts/results-apd.yaml
================================================
---
apiVersion: v1
kind: Namespace
metadata:
name: dashboard-apd
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: results-apd
namespace: dashboard-apd
labels:
dashboard: results-apd
spec:
replicas: 1
selector:
matchLabels:
dashboard: results-apd
template:
metadata:
labels:
dashboard: results-apd
spec:
containers:
- name: results-apd-container
image: nginx:alpine
ports:
- containerPort: 80
================================================
FILE: resources/staging-scripts/svc03-install-webserver.sh
================================================
#!/bin/bash
kubectl run -n kube-public server1-cka03-svcn --image=nginx --context=cluster3 --kubeconfig /root/.kube/clusters/cluster3_config
until [[ `kubectl get pods server1-cka03-svcn -n kube-public --kubeconfig /root/.kube/clusters/cluster3_config --context=cluster3 -o jsonpath='{.status.containerStatuses[].ready}'` == true ]]; do sleep 2; done;
#nohup kubectl --kubeconfig=/root/.kube/clusters/cluster2_config port-forward --address 0.0.0.0 -n kube-public server1-cka03-svcn 9999:80 &
cat < /lib/systemd/system/webserver.service
[Unit]
Description=kubectl proxy 8888
After=network.target
[Service]
User=root
ExecStart=/bin/bash -c "/usr/bin/kubectl --kubeconfig=/root/.kube/clusters/cluster3_config --context=cluster3 port-forward --address 0.0.0.0 -n kube-public server1-cka03-svcn 9999:80"
StartLimitInterval=0
RestartSec=10
Restart=always
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl enable webserver.service --now
kubectl --kubeconfig /root/.kube/clusters/cluster3_config --context=cluster3 apply -f - << EOF
apiVersion: v1
kind: Service
metadata:
name: external-webserver-cka03-svcn
spec:
ports:
- protocol: TCP
port: 80
targetPort: 9999
EOF
================================================
FILE: resources/staging-scripts/svn-01.yaml
================================================
apiVersion: v1
kind: Service
metadata:
creationTimestamp: null
name: curlme-cka01-svcn
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: curlme-ckaO1-svcn
status:
loadBalancer: {}
---
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: curlme-cka01-svcn
name: curlme-cka01-svcn
spec:
nodeName: cluster1-node01
containers:
- image: nginx:latest
name: curlme-cka01-svcn
ports:
- containerPort: 80
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
---
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: curlpod-cka01-svcn
name: curlpod-cka01-svcn
spec:
nodeName: cluster1-node02
containers:
- args:
- sleep
- "86400"
image: curlimages/curl
name: curlpod-cka01-svcn
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
================================================
FILE: resources/staging-scripts/svn-template-01.yaml
================================================
apiVersion: v1
kind: Service
metadata:
creationTimestamp: null
name: {{ Q1_SERVICE_NAME }}
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: {{ Q1_SERVICE_NAME }}
status:
loadBalancer: {}
---
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: {{ Q1_POD1_NAME }}
name: {{ Q1_POD1_NAME }}
spec:
nodeName: cluster1-node01
containers:
- image: nginx:latest
name: {{ Q1_POD1_NAME }}
ports:
- containerPort: 80
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
---
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: {{ Q1_POD2_NAME }}
name: {{ Q1_POD2_NAME }}
spec:
nodeName: cluster1-node02
containers:
- args:
- sleep
- "86400"
image: curlimages/curl
name: {{ Q1_POD2_NAME }}
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
================================================
FILE: resources/staging-scripts/test-v-apd14.yaml
================================================
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
stage: test01
test-app: v1
name: test-v1-apd
spec:
replicas: 3
selector:
matchLabels:
test-app: v1
template:
metadata:
labels:
test-app: v1
spec:
containers:
- image: kodekloud/webapp-color:v1
name: test-v1
---
apiVersion: v1
kind: Service
metadata:
labels:
tag: test-apd-svc
name: test-apd-svc
spec:
type: NodePort
ports:
- port: 8080
protocol: TCP
targetPort: 8080
selector:
test-app: v1
status:
loadBalancer: {}
================================================
FILE: resources/staging-scripts/update-ns-kubelet-cka01-svcn.sh
================================================
#!/bin/bash
ssh cluster1-controlplane "KUBE_DNS=\$(kubectl get svc -n kube-system kube-dns -o jsonpath='{.spec.clusterIP}'); \
ssh cluster1-node02 sed -i "s/\$KUBE_DNS/10.96.0.1/g" /var/lib/kubelet/config.yaml; \
ssh cluster1-node02 systemctl restart kubelet"
export KUBECONFIG=/root/.kube/clusters/cluster1_config
kubectl apply -f - << EOF
apiVersion: v1
kind: Service
metadata:
creationTimestamp: null
name: curlme-cka01-svcn
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: curlme-cka01-svcn
status:
loadBalancer: {}
---
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: curlme-cka01-svcn
name: curlme-cka01-svcn
spec:
nodeName: cluster1-node01
containers:
- image: nginx:latest
name: curlme-cka01-svcn
ports:
- containerPort: 80
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
---
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: curlpod-cka01-svcn
name: curlpod-cka01-svcn
spec:
nodeName: cluster1-node02
containers:
- args:
- sleep
- "86400"
image: curlimages/curl
name: curlpod-cka01-svcn
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
EOF
================================================
FILE: resources/staging-scripts/webapp-apd05.yaml
================================================
---
apiVersion: v1
kind: Namespace
metadata:
name: dev-apd
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: webapp-apd
name: webapp-apd
namespace: dev-apd
spec:
replicas: 2
selector:
matchLabels:
app: webapp-apd
template:
metadata:
labels:
app: webapp-apd
spec:
containers:
- image: kodekloud/webapp-color
name: webapp-color-apd
================================================
FILE: resources/staging-scripts/webapp-wear.yaml
================================================
---
kind: Namespace
apiVersion: v1
metadata:
name: app-space
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp-wear-cka09-svcn
namespace: app-space
spec:
replicas: 1
selector:
matchLabels:
app: webapp-wear-cka09-svcn
template:
metadata:
labels:
app: webapp-wear-cka09-svcn
spec:
containers:
- name: simple-webapp
image: kodekloud/ecommerce:apparels
imagePullPolicy: Always
ports:
- containerPort: 8080
================================================
FILE: resources/statging-cka16-trb-1.yaml
================================================
apiVersion: v1
kind: Service
metadata:
name: pink-svc-cka16-trb
spec:
ports:
- port: 5000
protocol: UDP
targetPort: 5000
selector:
app: pink-app-cka16-trb
type: ClusterIP
---
apiVersion: v1
kind: Pod
metadata:
name: pink-pod-cka16-trb
labels:
app: pink-app-cka16-trb
spec:
containers:
- name: app
image: alpine/socat
command:
- /bin/sh
- -c
- apk add curl > /dev/null 2>&1 ; socat -U TCP-LISTEN:5000,fork
EXEC:'curl example.com',stderr,pty,echo=0
ports:
- containerPort: 5000
================================================
FILE: resources/statging-cka16-trb-2.yaml
================================================
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: pink-ing-cka16-trb
spec:
ingressClassName: nginx
rules:
- host: kodekloud-pink.app
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: pink-svc-cka16-trb
port:
number: 5000
================================================
FILE: resources/stating-cluster2-cka26.yaml
================================================
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: logs-cka26-trb
namespace: kube-system
labels:
k8s-app: logs-cka26-trb
spec:
selector:
matchLabels:
name: logs-cka26-trb
template:
metadata:
labels:
name: logs-cka26-trb
spec:
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
containers:
- name: logs-cka26-trb
image: busybox
command:
- /bin/sh
- -c
- sleep 1000
resources:
limits:
memory: 50Mi
requests:
cpu: 10m
memory: 20Mi
volumeMounts:
- name: varlog
mountPath: /var/log
terminationGracePeriodSeconds: 30
volumes:
- name: varlog
hostPath:
path: /var/log
================================================
FILE: resources/trace-wl08.yaml
================================================
---
apiVersion: v1
kind: Namespace
metadata:
name: test-wl08
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: trace-wl08
name: trace-wl08
namespace: test-wl08
spec:
replicas: 1
selector:
matchLabels:
app: trace-wl08
template:
metadata:
labels:
app: trace-wl08
spec:
containers:
- command:
- sleep
- "5000"
image: busybox:1.28
name: busybox-wl08
================================================
FILE: resources/weave/weave-daemonset-k8s.yaml
================================================
apiVersion: v1
kind: List
items:
- apiVersion: v1
kind: ServiceAccount
metadata:
name: weave-net
labels:
name: weave-net
namespace: kube-system
- apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: weave-net
labels:
name: weave-net
rules:
- apiGroups:
- ''
resources:
- pods
- namespaces
- nodes
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- networkpolicies
verbs:
- get
- list
- watch
- apiGroups:
- 'networking.k8s.io'
resources:
- networkpolicies
verbs:
- get
- list
- watch
- apiGroups:
- ''
resources:
- nodes/status
verbs:
- patch
- update
- apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: weave-net
labels:
name: weave-net
roleRef:
kind: ClusterRole
name: weave-net
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
name: weave-net
namespace: kube-system
- apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: weave-net
namespace: kube-system
labels:
name: weave-net
rules:
- apiGroups:
- ''
resources:
- configmaps
resourceNames:
- weave-net
verbs:
- get
- update
- apiGroups:
- ''
resources:
- configmaps
verbs:
- create
- apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: weave-net
namespace: kube-system
labels:
name: weave-net
roleRef:
kind: Role
name: weave-net
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
name: weave-net
namespace: kube-system
- apiVersion: apps/v1
kind: DaemonSet
metadata:
name: weave-net
labels:
name: weave-net
namespace: kube-system
spec:
# Wait 5 seconds to let pod connect before rolling next pod
selector:
matchLabels:
name: weave-net
minReadySeconds: 5
template:
metadata:
labels:
name: weave-net
spec:
initContainers:
- name: weave-init
image: 'weaveworks/weave-kube:2.8.1'
command:
- /home/weave/init.sh
env:
securityContext:
privileged: true
volumeMounts:
- name: cni-bin
mountPath: /host/opt
- name: cni-bin2
mountPath: /host/home
- name: cni-conf
mountPath: /host/etc
- name: lib-modules
mountPath: /lib/modules
- name: xtables-lock
mountPath: /run/xtables.lock
readOnly: false
containers:
- name: weave
command:
- /home/weave/launch.sh
env:
- name: IPALLOC_RANGE
value: 10.32.1.0/24
- name: INIT_CONTAINER
value: "true"
- name: HOSTNAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
image: 'weaveworks/weave-kube:2.8.1'
readinessProbe:
httpGet:
host: 127.0.0.1
path: /status
port: 6784
resources:
requests:
cpu: 50m
securityContext:
privileged: true
volumeMounts:
- name: weavedb
mountPath: /weavedb
- name: dbus
mountPath: /host/var/lib/dbus
readOnly: true
- mountPath: /host/etc/machine-id
name: cni-machine-id
readOnly: true
- name: xtables-lock
mountPath: /run/xtables.lock
readOnly: false
- name: weave-npc
env:
- name: HOSTNAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
image: 'weaveworks/weave-npc:2.8.1'
#npc-args
resources:
requests:
cpu: 50m
securityContext:
privileged: true
volumeMounts:
- name: xtables-lock
mountPath: /run/xtables.lock
readOnly: false
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
hostPID: false
restartPolicy: Always
securityContext:
seLinuxOptions: {}
serviceAccountName: weave-net
tolerations:
- effect: NoSchedule
operator: Exists
- effect: NoExecute
operator: Exists
volumes:
- name: weavedb
hostPath:
path: /var/lib/weave
- name: cni-bin
hostPath:
path: /opt
- name: cni-bin2
hostPath:
path: /home
- name: cni-conf
hostPath:
path: /etc
- name: cni-machine-id
hostPath:
path: /etc/machine-id
- name: dbus
hostPath:
path: /var/lib/dbus
- name: lib-modules
hostPath:
path: /lib/modules
- name: xtables-lock
hostPath:
path: /run/xtables.lock
type: FileOrCreate
priorityClassName: system-node-critical
updateStrategy:
type: RollingUpdate
================================================
FILE: resources/webapp-color-wl10.yaml
================================================
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: webapp-color-wl10
name: webapp-color-wl10
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: webapp-color-wl10
template:
metadata:
labels:
app: webapp-color-wl10
spec:
containers:
- image: kodekloud/webapp-color
name: webapp-color-wl10
env:
- name: APP_COLOR
value: pink
---
apiVersion: v1
kind: Service
metadata:
name: webapp-color-wl10-service
namespace: default
spec:
ports:
- port: 8080
protocol: TCP
targetPort: 8080
selector:
app: webapp-color-wl10
type: NodePort
================================================
FILE: resources/webapp-pod-wl05.yaml
================================================
---
apiVersion: v1
kind: Namespace
metadata:
name: canara-wl05
---
apiVersion: v1
kind: Pod
metadata:
labels:
run: webapp-pod-wl05
name: webapp-pod-wl05
namespace: canara-wl05
spec:
containers:
- image: kodekloud/simple-webapp-mysql
name: webapp-pod-wl05
dnsPolicy: ClusterFirst
restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
labels:
app: webapp-svc-wl05
name: webapp-svc-wl05
namespace: canara-wl05
spec:
ports:
- name: 8080-8080
port: 8080
protocol: TCP
targetPort: 8080
nodePort: 31020
selector:
run: webapp-pod-wl05
type: NodePort
---
apiVersion: v1
kind: Pod
metadata:
namespace: canara-wl05
labels:
run: mysql-pod-wl05
name: mysql-wl05
spec:
containers:
- image: mysql:5.6
name: mysql-pod-wl05
env:
- name: MYSQL_ROOT_PASSWORD
value: password123
---
apiVersion: v1
kind: Service
metadata:
namespace: canara-wl05
labels:
app: mysql-svc-wl05
name: mysql-svc-wl05
spec:
ports:
- name: 3306-3306
port: 3306
protocol: TCP
targetPort: 3306
selector:
run: mysql-pod-wl05
type: ClusterIP
================================================
FILE: resources/webapp-wear-cka09-svcn.yaml
================================================
---
kind: Namespace
apiVersion: v1
metadata:
name: app-space
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: webapp-wear-cka09-svcn
namespace: app-space
spec:
replicas: 1
selector:
matchLabels:
app: webapp-wear-cka09-svcn
template:
metadata:
labels:
app: webapp-wear-cka09-svcn
spec:
containers:
- name: simple-webapp
image: kodekloud/ecommerce:apparels
imagePullPolicy: Always
ports:
- containerPort: 8080
================================================
FILE: resources/webapp-wl07.yaml
================================================
---
apiVersion: v1
kind: Namespace
metadata:
name: dev-wl07
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: webapp-wl07
name: webapp-wl07
namespace: dev-wl07
spec:
replicas: 3
selector:
matchLabels:
app: webapp-wl07
template:
metadata:
labels:
app: webapp-wl07
spec:
containers:
- image: kodekloud/webapp-color
name: webapp-color-wl07