```
* Install and run from a remote location:
```
go install github.com/bregman-arie/myApp
bin/myApp
```
* var definition
```
var i int = 42
OR
i := 42
```
* Define several variables
```
var (
name string = "hello"
num int = 42
```
================================================
FILE: resources/groovy.md
================================================
### Cheat Sheet
* Type of a variable
`println obj.getClass()`
================================================
FILE: resources/jenkins/README.md
================================================
# Jenkins
## Documentation
Title | Author | Comments
:------ |:------|:------:
[Official Jenkins Doc](https://jenkins.io/user-handbook.pdf) | jenkinsci-docs | |
[Job DSL Plugin](https://jenkinsci.github.io/job-dsl-plugin) | jenkinsci | |
## Tutorials & Articles
Title | Author | Comments
:------ |:------|:------:
[Jenkins Tutorial For Beginners](https://devopscube.com/jenkins-2-tutorials-getting-started-guide) | Focusing on installing and configuring Jenkins
[Top 10 Best Practices for Jenkins Pipeline Plugin](https://www.cloudbees.com/blog/top-10-best-practices-jenkins-pipeline-plugin) | apemberton | |
[The Need For Jenkins Pipeline](https://www.cloudbees.com/blog/need-jenkins-pipeline) | Viktor Farcic | Great article. Recommended.
[Jenkins Pipeline](https://www.cloudbees.com/blog/jenkins-pipeline) | Viktor Farcic | Another great article from Victor.
## Scripts
Repository | Comments
:------ |:------:
[pipeline-examples](https://github.com/jenkinsci/pipeline-examples) | The [HTML](https://jenkins.io/doc/pipeline/examples) version |
[This Repository](https://github.com/bregman-arie/jenkins-goodies/tree/master/scripts) | |
## Presentations
Title | Author | Comments
:------ |:------|:------:
[Jenkins API](https://www.slideshare.net/ArieBregman/jenkins-api) | Arie Bregman | |
## Blogs
Name | Comments
:------ |:------:
[Cloudbees Blog](https://www.cloudbees.com/blog) | |
[Jenkins Community Blog](https://jenkins.io/node/) | |
## Videos
Name | Comments
:------ |:------:
[Multi Branch Pipeline](https://www.youtube.com/watch?v=11z2x3VYO_I&t=) | For projects with multiple branches |
[How to create Jenkins Pipeline with an Example](https://www.youtube.com/watch?v=s73nhwYBtzE) | Great for beginners |
[Codifying the Build and Release Process with a Jenkins Pipeline Shared Library](https://www.youtube.com/watch?v=lzzx59kLW9w) |
## Checklist
- [ ] **Terms & Entities**
- [ ] Job
- [ ] Build
- [ ] Plugin
- [ ] Slave
- [ ] **Job**
- [ ] Freestyle
- [ ] Pipeline
- [ ] Declarative
- [ ] Scripted
- [ ] Checkpoint (cloudbees only at this point)
- [ ] **Slave**
- [ ] Agent
- [ ] Executor
- [ ] Label
### Cheat Sheet
* Set build result to FAILURE/SUCCESS/UNSTABLE: `currentBuild.result = 'FAILURE'`
================================================
FILE: resources/jenkins/jjb/gate.md
================================================
The following is the job definition of a gate to test changes to job definitions
- job:
name: 'gate-jenkins-jjb'
defaults: job_defaults
concurrent: true
parameters:
- string: { name: GERRIT_REFSPEC, default: +refs/heads/master }
- string: { name: GERRIT_BRANCH, default: origin/master }
- string: { name: DEBUG, default: '' }
description: |
Managed by Jenkins Job Builder. Do not edit via web.
Gate against changes to job definitions
node:
properties:
- build-discarder:
days-to-keep: 14
artifact-days-to-keep: 14
scm:
- codeng-scm:
project: '
branch: "master"
refspec: "$GERRIT_REFSPEC"
triggers:
- trigger-rhos-qe-jenkins:
file_paths:
- compare-type: 'REG_EXP'
pattern: '(jobs/.*|bin/.*|requirements\.txt)'
builders:
- shell: |
#!/bin/bash
set -euo pipefail
IFS=$'\n\t'
[ "${DEBUG:-0}" != 0 ] && set -x
PIP_LOG=$WORKSPACE/pip.log
tox -e jobs
publishers:
- archive:
artifacts: '**/*.log'
Also, add this in tox.ini:
[testenv:jobs]
commands =
{[testenv]commands}
mkdir -p out-xml
bash -c "XDG_CACHE_HOME={toxworkdir}/.jjb_cache jenkins-jobs --ignore-cache test --config-xml -o out-xml --recursive jobs 2>{envname}.log || \{ >&2 cat {envname}.log; exit 1; \}"
================================================
FILE: resources/jenkins/pipeline/README.md
================================================
## Pipeline in a nutshell
* A moment of history: Pipeline plugin was developed after the success
of Build Flow Plugin and it was first called 'Workflow Plugin'.
* It's considered to be better than freestyle type because it's implemented
in code, survives restarts and can be extended easily.
* Pipeline supports two syntaxes, Declarative and Scripted.
* [Here](https://github.com/bregman-arie/jenkins-goodies/blob/master/pipeline/basic_declarative_pipeline) is an example for simple declarative pipeline
* 'step' is a single task and tells Jenkins what to do.
* 'stage' is a collection of steps. Example for stages: 'Test', 'Build', 'Deploy'.
* 'node' allocates an executor and workspace for executing the steps in all the stages
* To use Jenkins pipeline, you need Jenkins 2 and Pipeline plugin installed.
* Scripted pipeline is written in Groovy.
* You can create a new pipeline by using Jenkins UI or adding a Jenkinsfile in the relevant repository. Using Jenkinsfile is recommended.
* To declare pipeline with UI -> click on 'new item', enter the name, choose 'Pipeline' from type options and click on 'save'.
* You can use https:///pipeline-syntax as built-in documentation for generating simple or complex
pipeline snippets.
* There are several types of variables supported in a pipeline. You can find updated reference on global variables
here: https:///pipeline-syntax/globals
* env variables are accessible from Groovy code with 'env.'.
* params are read-only variables exposed for you to use in the pipeline.
* currentBuild variables are set in the context of a specific build
* A reference of the available steps can be found [here](https://jenkins.io/doc/pipeline/steps).
There is also such reference in your Jenkins instance https:///pipeline-syntax/html
* Pipeline examples can be found [here](https://jenkins.io/doc/pipeline/examples)
================================================
FILE: resources/jenkins/pipeline/basic_declarative_pipeline
================================================
// Best practice is to actually call this file 'Jenkinsfile'
// I named it 'basic_pipeline' just to explain its purpose
// Pipeline is written in Pipeline DSL syntax
pipeline {
// Jenkins to allocate workspace and executor
agent any
// Stages are block of code you execute through your build
stages {
// This is the first stage the pipeline will execute
// A stage execute specific step of your flow
stage('Build') {
// Steps to be executed as part of the 'Build' stage
steps {
sh 'echo "Building"' // Execute the 'echo' shell command
}
}
// Once previous stage finished successfully, Jenkins will
// execute this stage.
stage('Test'){
steps {
sh 'echo "Testing"'
// To publish test results add this: junit 'reports/**/*.xml'
}
}
stage('Deploy') {
steps {
sh 'echo "Deploying"'
}
}
}
} // End of the pipeline
================================================
FILE: resources/jenkins/pipeline/basic_scripted_pipeline
================================================
#!/usr/bin/env groovy
================================================
FILE: resources/jenkins/pipeline/tips.md
================================================
# Pipeline - Tips
* Use https:///pipeline-syntax as built-in documentation for generating simple or complex
pipeline snippets.
* Use https:///pipeline-syntax/globals to find out what variables you can use in your pipeline.
* A reference of the available steps can be found [here](https://jenkins.io/doc/pipeline/steps).
There is also such reference in your Jenkins instance https:///pipeline-syntax/html
* Learn from [examples](https://jenkins.io/doc/pipeline/examples)
================================================
FILE: resources/jenkins/scripts/README.md
================================================
## How to run
There are several ways to run the snippets in this repository. The quickest would be:
1. Go to https:///script
2. Paste the script
3. Click on "Run"
4. Enjoy :)
================================================
FILE: resources/jenkins/scripts/builds/abort.groovy
================================================
Jenkins.instance.getItemByFullName("").getBuildByNumber().finish(hudson.model.Result.ABORTED, new java.io.IOException("Aborting build"));
================================================
FILE: resources/jenkins/scripts/builds/remove_old.groovy
================================================
// Remove old builds that are older than the number of given days
def removeOldBuilds(buildDirectory, days = 14) {
def wp = new File("${buildDirectory}")
def currentTime = new Date()
def backTime = currentTime - days
wp.list().each { fileName ->
folder = new File("${buildDirectory}/${fileName}")
if (folder.isDirectory()) {
def timeStamp = new Date(folder.lastModified())
if (timeStamp.before(backTime)) {
folder.delete()
}
}
}
}
return this;
================================================
FILE: resources/jenkins/scripts/builds/remove_old_v2.groovy
================================================
// Remove builds older than 5 days
builds = Jenkins.instance.getItem('test2').getBuilds()
def daysToKeep = 5
builds.each{ build ->
def runningSince = groovy.time.TimeCategory.minus( new Date(), build.getTime() ).getDays();
if (runningSince > daysToKeep){
build.delete();
println "Build " + build + "was deleted since it was built " + runningSince + " days ago";
}
}
================================================
FILE: resources/jenkins/scripts/jobs/delete_jobs_with_regex.groovy
================================================
def jobs = Jenkins.instance.items.findAll { job -> job.name =~ /the_regex/ }
jobs.each { job ->
println job.name
//job.delete()
}
================================================
FILE: resources/jenkins/scripts/jobs/print_all_jobs.groovy
================================================
jobs = jenkins.model.Jenkins.instance.getJobNames()
println "Number of jobs: " + jobs.size()
println "\nList of jobs:\n" + jobs.join("\n")
/*
Another version:
jobs = jenkins.model.Jenkins.instance.items
println "Number of jobs: " + jobs.size()
println "\nList of jobs:\n"
jobs.each { job -> println job.name }
*/
================================================
FILE: resources/jenkins/scripts/jobs/print_jobs_with_regex.groovy
================================================
jobs = jenkins.model.Jenkins.instance.items.findAll { job -> job.name =~ /network.*10_((?!SRIOV).)*$/ }
println "Number of jobs: " + jobs.size() + "\n"
jobs.each { job -> println job.name }
================================================
FILE: resources/jenkins/scripts/jobs/rename_job.groovy
================================================
def job = jenkins.model.Jenkins.instance.getItem('dashboard');
def currentName = job.name;
def newName = 'test2';
job.renameTo(newName);
println "\nRenamed " + currentName + " to " + newName;
================================================
FILE: resources/jenkins/scripts/jobs/rename_jobs_with_regex.groovy
================================================
def jobs = Jenkins.instance.items.findAll { job -> job.name =~ /the_regex/ }
def newNameAddition = 'test2'
jobs.each { job ->
currentName = job.name
job.renameTo(job.name + newNameAddition);
println "\nRenamed " + currentName + " to " + currentName + newNameAddition + "\n";
}
================================================
FILE: resources/jenkins/scripts/nodes/check_all_online.groovy
================================================
// Check if all slaves are online. Fail the script if not.
int excode = 0
for (slave in hudson.model.Hudson.instance.slaves) {
if (slave.getComputer().isOffline().toString() == "true"){
println('The Slave ' + slave.name + " is offline");
excode++;
}
}
if (excode > 0) {
println("We have a Slave down! I repeat, we have a slave down!");
return 1;
}
================================================
FILE: resources/jenkins/scripts/nodes/print_node_groups.groovy
================================================
import jenkins.model.Jenkins;
def list = []
def description = ''
def slave_label_map = [:]
for (slave in Jenkins.instance.slaves) {
description = slave.getNodeDescription()
if (description.contains('<#{') && description.contains('}#>')) {
words = slave.getLabelString().split()
def labelListForSlave = []
words.each() {
labelListForSlave.add(it);
list.add(it)
}
slave_label_map.put(slave.name, labelListForSlave)
}
}
list.unique()
================================================
FILE: resources/jenkins/scripts/notification/notifier/__init__.py
================================================
# Copyright 2018 Arie Bregman
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
================================================
FILE: resources/jenkins/scripts/notification/notifier/main.py
================================================
# Copyright 2018 Arie Bregman
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import json
import requests
import sys
from notifier import parse
message = 'Build {}'.format('FAILED')
headers = {
'Content-Type': 'application/json',
}
payload = {
'message': message,
}
endpoint = ''
def main():
"""Main entry point."""
# Parse given arguments
parser = parse.create_parser()
args = parser.parse_args()
r = requests.post(endpoint,
data=json.dumps(payload + args),
headers=headers)
print(r.status)
if __name__ == '__main__':
sys.exit(main())
================================================
FILE: resources/jenkins/scripts/notification/notifier/parse.py
================================================
# Copyright 2018 Arie Bregman
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import argparse
def create_parser():
"""Returns argument parser"""
parser = argparse.ArgumentParser(add_help=True)
parser.add_argument('--debug', '-d', action='store_true',
dest="debug", help='Turn on debug')
parser.add_argument('--yaml-file', '-y', dest='yaml_f', nargs='+',
help='YAML files')
return parser
================================================
FILE: resources/jenkins/scripts/plugins/list_all_plugins.groovy
================================================
Jenkins.instance.pluginManager.plugins.sort { it.getShortName() }.each{ plugin -> println "${plugin.getShortName()}:${plugin.getVersion()}" }
================================================
FILE: resources/jenkins/scripts/tests/count_test_failures.groovy
================================================
import jenkins.*
import jenkins.model.*
import hudson.*
import hudson.model.*
def jobNameStartsWith = 'DFG-network-neutron'
def textToMatch = "Traceback"
Jenkins.instance.items.findAll { it.displayName.startsWith(jobNameStartsWith) }
.each { job ->
if(job.builds) {
def build = job.isBuilding() ? job.builds[1] : job.builds[0]
def log = build.logFile.text
def regexPattern = "(?i).*${textToMatch}.*"
def summary = log =~ /${regexPattern}/
println "${summary.size()}: $job.displayName"
}
}
return ""
================================================
FILE: resources/jinja2.md
================================================
### CheatSheet
* Split string to list of items using comma as the separator: `{% set list1 = variable1.split(',') %}`
================================================
FILE: resources/kubernetes.md
================================================
## Kubernetes
### Getting started with Kubernetes
Name | Comments
:------ |:--------:
[kubernetes.io](https://kubernetes.io) | Official Kubernetes site by Google
[Kubernetes 101](https://medium.com/google-cloud/kubernetes-101-pods-nodes-containers-and-clusters-c1509e409e16) | Great beginner article on Kubernetes fundamental concepts
[Kubernetes Tutorial for Beginners](https://www.youtube.com/watch?v=X48VuDVv0do&ab_channel=TechWorldwithNana) | Full video of 4 hours on Kubernetes (2020)
[Learning Path: Kubernetes](https://developer.ibm.com/series/kubernetes-learning-path/) | From basic to advanced Kubernetes learning series
[Kubernetes 101 - Concepts and Why It Matters](https://www.magalix.com/blog/kubernetes-101-concepts-and-why-it-matters?fbclid=IwAR10FZlZ9Pw5c94tGRlgsCrVZTa1bSV2mbxEP8p4cXZ5T-k4VXF-3OUKkFo) |
[kubernetes-workshop](https://github.com/eon01/kubernetes-workshop) |
[Kubernetes Deployment Tutorial](https://devopscube.com/kubernetes-deployment-tutorial) |
[Katacoda](https://www.katacoda.com/courses/kubernetes) | Learn Kubernetes using Interactive Browser-Based Scenarios
[Kubernetes Scheduler](https://jvns.ca/blog/2017/07/27/how-does-the-kubernetes-scheduler-work/) | How does the Kubernetes scheduler work?
### Kubernetes - Deep Dive
Name | Comments
:------ |:--------:
[Kubernetes Networking](https://github.com/nleiva/kubernetes-networking-links) | Kubernetes Networking Resources
[Liveness and Readiness Probes](https://www.openshift.com/blog/liveness-and-readiness-probes) |
### Kubernetes - Troubleshooting
Name | Comments
:------ |:--------:
[troubleshoot.sh](https://troubleshoot.sh) | "A kubectl plugin providing diagnostic tools for Kubernetes applications"
[Kubernetes Troubleshooting Visual Guide](https://learnk8s.io/troubleshooting-deployments) |
### Kubernetes - Security
Name | Comments
:------ |:--------:
[Kubescape](https://github.com/armosec/kubescape) | "Kubescape is the first tool for testing if Kubernetes is deployed securely as defined in Kubernetes Hardening Guidance by NSA and CISA"
[Falco](https://falco.org) | "Falco...is the de facto Kubernetes threat detection engine"
### Kubernetes - Misc
Name | Comments
:------ |:--------:
[confTest](https://www.conftest.dev) | "Conftest is a utility to help you write tests against structured configuration data" (Used in the development phase)
[datree](https://www.datree.io) | "Prevent Kubernetes Misconfigurations From Reaching Production" (Used in development phase)
[gatekeeper](https://github.com/open-policy-agent/gatekeeper) | Used in the production
[telepresence](https://www.telepresence.io) | "FAST, LOCAL DEVELOPMENT FOR KUBERNETES AND OPENSHIFT MICROSERVICES"
[Kubernetes CheatSheet](https://cheatsheet.dennyzhang.com/cheatsheet-kubernetes-A4) |
[OperatiorHub.io](https://www.operatorhub.io) | Kubernetes native applications
[YAML templates](https://cheatsheet.dennyzhang.com/kubernetes-yaml-templates) |
[Kubesort](https://github.com/AATHITH/kubesort) | "kubesort helps you sort the results from kubectl get in an easy way"
[IngressMonitorController](https://github.com/stakater/IngressMonitorController) | "A Kubernetes controller to watch ingresses and create liveness alerts for your apps/microservices"
### Kubernetes - SRE
Name | Comments
:------ |:--------:
[KubeInvaders](https://github.com/lucky-sideburn/KubeInvaders) | "Chaos Engineering Tool for Kubernetes and Openshift"
### Kubernetes - Certificates
Name | Comments
:------ |:--------:
[CKAD-Practice-Questions](https://github.com/bbachi/CKAD-Practice-Questions) | "a consolidated list for CKAD practice questions"
[CKAD Prep Exam Video](https://www.youtube.com/watch?v=TPXwVmvzlV4&ab_channel=TheFrontOpsGuy) | A video of doing a CKAD prep exam (2020)
[CKA Complete Prep Exam ](https://github.com/walidshaari/Kubernetes-Certified-Administrator) | CKA prep exam (2021)
[CKAD Complete Prep Exam ](https://github.com/dgkanatsios/CKAD-exercises) | CKAD prep exam (2021)
[CKS Complete Prep Exam ](https://github.com/walidshaari/Certified-Kubernetes-Security-Specialist) | CKA prep exam (2021)
### Kubernetes - Labs
Name | Comments
:------ |:--------:
[Building a Kubernetes 1.23 Cluster with Kubeadm](https://github.com/croguerrero/CKA-Labs/blob/main/resources/KC.md) | "Labs CKA - Build a Kubernetes cluster 1.23"
### Best Practices
#### Security Best Practices
* Secure inter-service communication (one way is to use Istio to provide mutual TLS)
* Isolate different resources into separate namespaces based on some logical groups
* Use supported container runtime (if you use Docker then drop it because it's deprecated. You might want to CRI-O as an engine and podman for CLI)
* Test properly changes to the cluster (e.g. consider using Datree to prevent kubernetes misconfigurations)
* Limit who can do what (by using for example OPA gatekeeper) in the cluster
* Use NetworkPolicy to apply network security
* Consider using tools (e.g. Falco) for monitoring threats
### CheatSheet
#### Minikube
* Minikube version: `minikube version`
* Start cluster: `minikube start`
* Delete cluster: `minikube delete`
#### Common Kubectl Operations
* Create objects defined in a YAML: kubectl apply -f rs.yaml
#### Service Accounts
* List service accounts: `kubectl get serviceaccounts`
### Cluster
* Cluster version: `kubectl version`
* Cluster information: `kubectl cluster-info`
* List nodes: `kubectl get nodes`
### Images
* List all containers images from all namespaces:
```
kubectl get pods --all-namespaces -o jsonpath="{.items[*].spec.containers[*].image}" |\
tr -s '[[:space:]]' '\n' |\
sort |\
uniq -c
```
### Pods
* List of Pods in current namespace: `kubectl get po`
* List of Pods in all amespaces: `kubectl get po --all-namespaces`
* Get containers names: `kubectl get po -o jsonpath="{.spec.containers[*].name}"`
* Create a Pod from file: `kubectl create -f pod_definition.yaml`
* Delete a Pod using a YAML definition: `kubectl delete -f pod_definition.yaml`
* Delete a Pod using the Pod name: `kubectl delete `
* Delete a Pod instantly: `kubectl delete --grace-period=0 --force`
* Execute commands inside a container: `kubectl exec -it -c ls`
* Display logs of a Pod: `kubectl logs `
* Display logs of a specific container in a Pod: `kubectl logs -c `
* Get Pod name based on specific labels
```
POD_NAME=$(kubectl get pod \
--no-headers \
-o=custom-columns=NAME:.metadata.name \
-l type=api,service=some-service \
| tail -1)
```
* Get pods sorted by time creation: `kubectl get pods --sort-by='{.metadata.creationTimestamp}'`
* Run long/endless running container: `kubectl run test --image ubuntu -- sleep 2000000000`
* Get a shell access to a container: `kubectl exec --stdin --tty some-shell -- /bin/bash`
### User
* Creating a new user
```
openssl genrsa -out user.key 2048 # create key
openssl req key user.key user.csr -subj "/CN=user /O=sgroup" # create csr
openssl x509 -req -in user.csr -CA ca.crt -CAkey ca.key -CAcreateseral -out user.crt -days 365
kubectl config set-credentials myuser --client-certificates=$PWD/user.crt --client-key=$PWD/user.key
kubectl config set-context myuser-context --cluster=k8s-cluster --user=user
```
### Service
* Expose a ReplicaSet: `kubectl expose rs REPLICASET_NAME --name=SERVICE_NAME --target-port=PORT --type=NodePort/SOME_OTHER_SERVICE_TYPE`
### Deployment
* Create and delete a deployment in one step:
```
$ kubectl create deployment demo --image=cloudnatived/demo:hello
$ kubectl get pods --selector app=demo
$ kubectl delete pods --selector app=demo
$ kubectl delete all --selector app=demo
```
### Aliases
* Use kubectl aliases to speed up and reduce typo errors, practice these alaises early at your work and study for the exam. some example aliases:
```
alias k='kubectl'
alias kg='kubectl get'
alias kgpo='kubectl get pod'
alias kcpyd='kubectl create pod -o yaml --dry-run=client'
alias ksysgpo='kubectl --namespace=kube-system get pod'
alias kd='kubectl delete'
alias kdf='kubectl delete -f'
## for quick deletes you can add --force --grace-period=0 **Not sure if it is a good idea if you are in a production cluster**
alias krmgf='kubectl delete --grace-period 0 --force'
alias kgsvcoyaml='kubectl get service -o=yaml'
alias kgsvcwn='watch kubectl get service --namespace'
alias kgsvcslwn='watch kubectl get service --show-labels --namespace'
#example usage of aliases
krmgf nginx-8jk71 # kill pod nginx-8jk71 using grace period 0 and force
```
### Monitoring
* Install metrics-server using Minikube: `minikube addons enable metrics-server`
### Secrets
* Install Bitnami Sealed secrets controller
```
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm repo update
helm install sealed-secrets-controller sealed-secrets/sealed-secrets
```
* sealed-secrets binary: `https://github.com/bitnami-labs/sealed-secrets/releases`
* encrypt secret in yaml or json format: `kubeseal -n my-namespace <.db-creds.yml> db-creds.json`
* apply secret on the cluser: `kubectl apply -f db-creds.json -n my-namespace`
### Kustomize
* Generate customized YAML file(s): `kustomize build name_of_application`
* Apply generated customized YAML file(s) to the cluster: `kustomize build name_of_application`
================================================
FILE: resources/linux.md
================================================
# Linux
## Learn Linux - Tutorials and Guides
Name | Comments
:------|:------:
[Linux Journey](https://linuxjourney.com) | Written guides + exercises + quiz
[Techmint Linux](https://www.tecmint.com/free-online-linux-learning-guide-for-beginners) | Written articles/lessons
[Linux Survival](https://linuxsurvival.com/linux-tutorial-introduction) | Interactive guide
[NixCraft](https://www.cyberciti.biz/) | Succinct guides
[Linux Filesystem Explained](https://www.linux.com/training-tutorials/linux-filesystem-explained) | NSIA
[Linux SysOps Handbook](https://abarrak.gitbook.io/linux-sysops-handbook) | A study notes book for the common knowledge and tasks of a Linux system admin.
[LinuxToday](https://www.linuxtoday.com) | Contributor-driven news resources
## Linux Internals - Articles
Name | Comments
:------|:------:
[How are Unix pipes implemented?](https://toroid.org/unix-pipe-implementation) |
## Linux Internals - Videos
Name | Comments
:------|:------:
[Understanding fork() system call for new process creation](https://www.youtube.com/watch?v=PwxTbksJ2fo) |
## Learn Linux - Videos
Name | Comments
:------|:------:
[learnlinux.tv](https://www.learnlinux.tv) | Very practical videos on various topics
## Books
Author | Title | Comments
:------ |:------:|:--------:
William Shotts | [The Linux Command Line](http://linuxcommand.org/tlcl.php) | Free |
Christopher Negus | [Linux Bible](https://www.wiley.com/en-us/Linux+Bible%2C+9th+Edition-p-9781118999875) | |
## Sites
Name | Comments
:------|:------:
[Linux From Scratch](http://www.linuxfromscratch.org) |
[Operating Systems: Three Easy Pieces](http://pages.cs.wisc.edu/~remzi/OSTEP) |
## Videos
Presenter | Video | Length | Level
:------ |:------|:------:|:--------:
Joseph Delgadillo | [The Complete Linux Course: Beginner to Power User!](https://www.youtube.com/watch?v=wBp0Rb-ZJak&t=6578s) | 07:23:52 | Beginner |
## Tweak OS
Fedora disable alert sound
`dconf write /org/gnome/desktop/sound/event-sounds "false"`
## Presentations
Author | Presentation | Level | Comments
:------ |:------|:--------:|:--------:
Arie Bregman | [Linux Networking](https://www.slideshare.net/ArieBregman/linux-networking-113100224) | Beginner | |
Andrew McNico | [tcpdump](https://www.slideshare.net/j0b1n/tcpdump-hunter?qid=b71dea53-7829-40a3-b82b-4a669383eac6) | Beginner | |
## Cheat Sheet
### Common Operations
* Copy files to a remote host: `rsync -azv dir USER:REMOTE_HOST_ADDR`
* View file size: `du -sh DIR/FILE`
### Users
* Who is connected: `w`
### Troubleshooting and Debugging
* see list of processes and the CPU and memory the consume: `top`
* show how long the server is up running: `uptime`
* List open files: `lsof`
* List open connections: `lsof -i`
### SElinux
Get SElinux denials
```
semodule -DB
sudo ausearch -m avc -m user_avc -m selinux_err -m user_selinux_err -i -ts today
```
### Communication
* Broadcast a message to everyone: `wall`
### Tmux
* Join a session `tmux a`
* Attach to existing session: `tmux attach -t `
* New tab: `ctrl + B + c`
* New session called "bla": `tmux new -s bla`
#### Virsh
* Destroy: `virsh destroy `
* Shutdown: `virsh shutdown `
#### Date
* year: `date +”%Y”`
* month: `date +”%m”`
* day: `date +”%a”`
#### Fedora
* Reset "interface text" value: `gsettings reset org.gnome.desktop.interface font-name`
* Reset "monospace text" value: `gsettings reset org.gnome.desktop.interface monospace-font-name`
* Reset "document text" value: `gsettings reset org.gnome.desktop.interface document-font-name`
* Reset "Legacy Window Titles": `gsettings reset org.gnome.desktop.wm.preferences titlebar-font`
#### Files
* Sort files by size: `ls -l | sort -nk5`
* Find broken links: `find /some/path -type l -exec test ! -e {} \; -print`
### LDAP
* LDAP search for a username: `ldapsearch -x -W -D username`
* LDAP search for a username specifying DC and ldap server: `ldapsearch -x -H ldap://ldap.blabla.com -LLL -b 'dc=DOMAIN,dc=com' "rhatNickName=abregman"`
### YAML
* Validate YAML file with Ruby: `ruby -ryaml -e "p YAML.load(STDIN.read)" < some_file.yaml`
* Validate YAML file with Python: `pip install pyyaml; python -c 'import yaml, sys; print(yaml.safe_load(sys.stdin))' < some_file.yaml`
### Misc
* Random string of 7 characters: `mkpasswd -l 8`
* Generate 8 digit random number: `shuf -i 9999999-99999999 -n 1`
## Checklist
Check your Linux educational progress with the following list:
- [ ] **File System Hierarchy**
- [ ] `/` (aka root)
- [ ] `/bin` and `/sbin`
- [ ] `/opt`
- [ ] `/usr`
- [ ] `/var`
- [ ] `/home`
- [ ] `/etc`
- [ ] `/proc`
- [ ] `/tmp`
- [ ] Test
- [ ] What is '/'?
- [ ] What is stored in '/bin'? what about `/etc`?
- [ ] What '/home' is usually used for?
- [ ] What type of files can you find in '/var'?
- [ ] What is unique about `/tmp`?
- [ ] **Navigation**
- [ ] Commands
- [ ] `pwd` - where am I?
- [ ] `cd` - change directory
- [ ] relative vs. absolute paths
- [ ] Test
- [ ] How to check your current path?
- [ ] Where `cd ..` will take you?
- [ ] What would be the effect of running `cd .`?
- [ ] What will happen when running 'cd -'?
- [ ] **Files**
- [ ] Types of files
- [ ] Regular
- [ ] Directory
- [ ] Socket
- [ ] Block
- [ ] Link
- [ ] Commands
- [ ] `ls` - list files and directories
- [ ] `-a` for listing hidden files
- [ ] `-l` for list formt
- [ ] `-t` order by time
- [ ] `-F` better distinguish between regular files and directories
- [ ] `touch` - creating files (original intention is updating timestamp)
- [ ] nice to know: `touch file{1..5}`
- [ ] `rm` - remove files
- [ ] `-r` for recursive
- [ ] `-f` to force removal, no questions asked
- [ ] `mkdir` - create directories
- [ ] `-p` - for creating multiple nested directories
- [ ] `rmdir` - remove directories
- [ ] `echo` - display a line of text
- [ ] `cat` - concatenate files (common usage: read a file)
- [ ] `mv` - move files directories (also rename files and directories)
- [ ] `cp` - copy a file
- [ ] `-r` for recursive (copy a directory)
- [ ] Test
- [ ] How to list hidden files?
- [ ] How to create an empty new file?
- [ ] How to remove a directory?
- [ ] How to rename a file?
- [ ] How to copy an entire directory with all its files?
- [ ] Commands
- [ ] `man` - manual for commands
- [ ] `which` - get full path for given command
- [ ] `whatis` - one-line manual page descriptions
- [ ] **I/O redirection**
- [ ] File Descriptor
- [ ] stdin 0 (input) <
- [ ] stdout 1 (output) >
- [ ] stderr 2 (error) 2>
- [ ] Append >>
- [ ] Test
- [ ] How to redirect output?
- [ ] What would be the result of the following command `blop 2> file`?
- [ ] Text Editor (one is enough)
- [ ] vim
- [ ] `i` to start typing
- [ ] `:wq` to exit (or `shit+zz`)
- [ ] nano
- [ ] emacs
- [ ] atom
- [ ] sublime
- [ ] Test (mainly for vim)
- [ ] How to remove an entire line?
- [ ] How to copy 5 lines?
- [ ] How to jump to the end of the line
- [ ] How to remove one word
- [ ] How to jump to the end of the file
- [ ] **Users**
- [ ] Commands
- [ ] useradd
- [ ] usermod
- [ ] userdel
- [ ] who
- [ ] Nice to know commands
- [ ] lastlog
- [ ] Test
- [ ] how to add a new user?
- [ ] should you be using your user or root?
- [ ] **Network**
- [ ] netstat
- [ ] **Monitoring and performence commands**
- [ ] top
- [ ] stat
- [ ] **Processes**
- [ ] Running in ackground (&)
- [ ] **Archives**
- [ ] what is it good for
- [ ] tar
- [ ] zip
- [ ] commands
- [ ] create an archive
- [ ] list archive's content
- [ ] remove an archive
- [ ] **Storage, Filesystem**
- [ ] inode
================================================
FILE: resources/machine_learning.md
================================================
# Machine Learning
## Checkbox List
- [ ] **Basic Terms**
- [ ] Sample
- [ ] Attribute
- [ ] **Data Mining**
- [ ] Association Rules
- [ ] Apriori
- [ ] Support
- [ ] Confidence
- [ ] Conviction
- [ ] Lift
- [ ] Eclat
- [ ] FP-growth
## Videos
Name | Description
:------|:------:
[Apriori](https://www.youtube.com/watch?v=WGlMlS_Yydk) | Apriori algorithm beginner explanation
[Bayesian Networks](https://www.youtube.com/watch?v=TuGDMj43ehw) | Bayesian beginner explanation
## Repos
Name | Description
:------|:------:
[PracticalAI](https://github.com/GokuMohandas/practicalAI) | A practical approach to learning and using machine learning.
================================================
FILE: resources/mongo.md
================================================
# MongoDB
## Tutorials
Name | Description
:------|:------:
[Guru99 MongoDB Tutorial](https://www.guru99.com/what-is-mongodb.html) | MongoDB Tutorial
[Tutorialspoint Tutorial](https://www.tutorialspoint.com/mongodb) | MongoDB Tutorial
## Authentication and Authorization
Name | Description
:------|:------:
[Tutorial User Administration Without You Integrating LDAP with MongoDB](https://www.youtube.com/watch?v=ky8OKWPCZXs&ab_channel=MongoDB) | Full explanation on Mongo and LDAP integration
## Cheat Sheet
### Mongo Shell
#### General
* DB stats: `db.stats()`
* Switch to a DB: `use DB_NAME`
* Show collections: `show collections`
* Show all the document in a collection: `db.COLLECTION_NAME.find({})`
* Count the number of documents in a collection: `db.COLLECTION_NAME.count()`
* Remove all the documents from a collection: `db.COLLECTION_NAME.remove({})`
* Remove field from all documents: `db.COLLECTION_NAME.update({}, {$unset: { FIELD_NAME: ""}}, {multi:true})`
* Rename field: ```db.hosts.updateMany( {}, { $rename: { "CURRENT_FIELD_NAME": "NEW_FIELD_NAME" } } )```
#### Mongoexport
* Export Mongo DB collection to a JSON file: `mongoexport --db=DB_NAME --collection=COLLECTION_NAME --out=data.json`
* Export specific fields from a Mongo DB collection to a JSON file: `mongoexport --db=DB_NAME --fields=FIELD_NAME,FIELD_NAME2 --collection=COLLECTION_NAME --out=data.json`
#### Aggregations
* Count the number of documents: `db.accounts.aggregate([ { "$count": "number of documents" }])`
* Show only the sub element 'attachment_uuids': `db.accounts.aggregate( [ {$project: {"hosts.attachment_uuids": 1, _id:0}} ] )`
#### Users
* Display users:
```
use admin
db.getUsers()
```
* Create administrative user
```
use admin
db.createUser({user: "admin", pwd: "some_password", roles: ["root"]})
```
* Authentice: `db.auth(USERNAME, PASSWORD)`
#### Roles
* Display roles
```
use admin
db.getRoles()
```
* Drop/Remove role: `db.dropRole("NAME_OF_THE_ROLE")`
#### LDAP
* Mongo Conf LDAP configuration
```
security:
# authorization: "enabled"
ldap:
servers: 'some.ldap.server.com'
userToDNMapping:
'[
{
match: "(.+)",
ldapQuery: "DC=X,DC=com??sub?(uid={0})"
}
]'
authz:
queryTemplate: "{USER}?memberOf?base"
setParameter:
authenticationMechanisms: 'PLAIN,SCRAM-SHA-1'
```
* Create role where the LDAP group "unicorns" has permissions to only run 'find' on "hosts" collection in 'test' db:
```
db.createRole({role:"cn=unicorns,ou=groups,dc=X,dc=com", privileges: [{ resource: {db: "", collection: "hosts"}, actions: ["find"]}], roles: [{ role: "read", db: "test"}]})
```
* Drop/Remove role from the previous example: `db.dropRole("cn=unicorns,ou=groups,dc=X,dc=com")`
### Mongo Python
* Add new element to an arrray only if it doesn't exists already
```
mongo_client.update(
{ "account_number": account_num },
{ "$addToSet": { "hosts": {'hostname': hostname} } }
)
```
* Add a new element to an array in sub-document (if it doesn't exists already):
```
mongo_client.update_one(
{ "account_number": account_num, "hosts.hostname": hostname },
{ "$addToSet": { "hosts.$.attachment_uuids": attachment_uuid } }
)
```
* Add all the elements in an array to an array in document (This is not about adding the array as it is but adding the elements of it):
```
mongo_client.update_one(
{'_id': 1},
{'$addToSet': {'some_list': { '$each': [1, 2, 3]} }}, upsert=True)
```
================================================
FILE: resources/monitoring.md
================================================
# Monitoring
## Articles
Name | Comments
:------|:------:
[Stack Overflow: How We Do Monitoring - 2018 Edition](https://nickcraver.com/blog/2018/11/29/stack-overflow-how-we-do-monitoring) |
## CLI Tools
Name | Comments
:------|:------:
[statuscakectl](https://github.com/omerh/statuscakectl) | Statuscake CLI
================================================
FILE: resources/network.md
================================================
# Network
## Docs
Name | Comments
:------|:------:
[TCP/IP networking referen](http://www.penguintutor.com/linux/basic-network-reference)
## Books
Author | Title | Comments
:------ |:------:|:--------:
James Kurose & Keith Ross | Computer Networking: A Top-Down Approach | Great for beginners
Andrew S. Tanenbaum | Computer Networks | Commonly used by academic networking courses
Richard Stevens | TCP/IP illustrated |
Richard Stevens | UNIX Network Programming |
OccupyTheWeb | Linux Basics for Hackers: Getting Started with Networking, Scripting, and Security in Kali | Includes a chapter on wireless networking
Evi Nemeth & Garth Snyder & Trent R. Hein & Ben Whaley & Dan Mackin | UNIX and Linux System Administration Handbook | A great book on Linux and there is a dedicated chapter for Linux networking
## Videos - Basic Networking Concepts
Presenter | Video | Length | Level
:------ |:------|:------:|:--------:
Rene Molenaar | [Address Resolution Protocol (ARP) Explained](https://www.youtube.com/watch?v=xTOyZ6TWQdM) | 00:05:13 | Beginner |
Eli the Computer Guy | [The OSI Model Demystified](https://www.youtube.com/watch?v=HEEnLZV2wGI) | 00:18:40 | Beginner |
Pieter De Decker | [Hub, Switch or Router? Network Devices Explained](https://www.youtube.com/watch?v=Ofjsh_E4HFY) | 00:07:39 | Beginner |
Pieter De Decker | [How Network Address Translation Works](https://www.youtube.com/watch?v=QBqPzHEDzvo) | 00:10:04 | Beginner |
Pieter De Decker | [How DHCP Works](https://www.youtube.com/watch?v=RUZohsAxPxQ) | 00:07:55 | Beginner |
Pieter De Decker | [UDP and TCP: Comparison of Transport Protocols](https://www.youtube.com/watch?v=Vdc8TCESIg8) | 00:11:34 | Beginner |
Ricky Tapia | [CSMA/CD](https://www.youtube.com/watch?v=IAKncL67Pp4) | 00:04:01 | Beginner |
## Videos - Linux
Presenter | Video | Length | Level
:------ |:------|:------:|:--------:
David Mahler | [Introduction to Linux Network Namespaces](https://www.youtube.com/watch?v=_WgUwUf1d34) | 00:25:50 | Beginner |
## Videos - OpenvSwitch
Presenter | Video | Slides | Length | Level
:------ |:------|:------:|:--------:|:--------:
David Mahler | [Introduction to Open vSwitch](https://www.youtube.com/watch?v=rYW7kQRyUvA) | | 00:14:35 | Beginner |
David Mahler | [An Introduction to Open vSwitch](https://www.youtube.com/watch?v=_PCRNUB7oNw) | [Slides](http://openvswitch.org/slides/openvswitch.en-2.pdf)| 00:35:49 | First half beginners, second half intermediate level |
## Videos - OpenFlow
Presenter | Video | Length | Level | Comments
:------ |:------|:------:|:--------:|:--------:
David Mahler | [Introduction to OpenFlow](https://www.youtube.com/watch?v=l25Ukkmk6Sk) | 00:15:25 | Beginner | |
Steven Wallace | [Describes the OpenFlow flow table](https://www.youtube.com/watch?v=-xLQHld3fPI) | 00:03:00 | Beginner | |
David Mahler | [OpenFlow with multiple Flow Tables](https://www.youtube.com/watch?v=TD5wmoD7XOE) | 00:09:30 | Intermediate | |
David Mahler | [OpenFlow flow entries on Open vSwitch (OVS)](https://www.youtube.com/watch?v=FyV4MoQ3T0I) | 00:14:22 | Intermediate | Required Knowledge: Open vSwitch |
## Videos - OpenStack Neutron
Presenter | Video | Length | Level | Comments
:------ |:------|:------:|:--------:|:--------:
[Shalu George & Naveen Joy] | [Neutron networking demystified for beginners](https://www.youtube.com/watch?v=c-x90PZFv8k) | 00:39:53 | Beginner | 2017 |
David Mahler | [Introduction to OpenStack Neutron](https://www.youtube.com/watch?v=yqFpyubsYfE) | 00:19:38 | Beginner | |
Assaf Muller | [Introduction to OpenStack Neutron](https://www.youtube.com/watch?v=IGGgVuZe7UA) | 00:33:20 | Beginner | |
Rossella Sblendido | [I Can't Ping My VM! Learn How to Debug Neutron and Solve Common Problems](https://www.youtube.com/watch?v=aNA8Pvewu2M) | 00:42:16 | Intermediate | Required Knowledge: neutron basics |
David Mahler | [OpenStack Neutron Packet Walkthrough (DVR)](https://www.youtube.com/watch?v=7IXEtUEZslg) | 00:44:56 | Intermediate | Required Knowledge: neutron, openvswitch, namespaces and openflow |
Mohamed Elsakhawy | [Neutron- Where Did My Packets Go?](https://www.youtube.com/watch?v=uKgMp5c6R-4) | 00:39:32 | Beginner | |
## Checklist - Network Newbie
- [ ] **OSI model**
- [ ] What layers the OSI model has and what is the purpose of each one of them?
- [ ] **Ethernet**
- [ ] What is CSMA/CD and how it works?
- [ ] **MAC address**
- [ ] What is it used for? Is it unique?
- [ ] How many bits are in a MAC address?
- [ ] **IP address**
- [ ] What is it used for? Is it unique worldwide?
- [ ] What are the two parts of an IP address?
- [ ] How many bits are in an IP address?
- [ ] **Subnets**
- [ ] Two hosts are on the same subnet if they have the same '_____'
- [ ] What is a netmask?
- [ ] What two syntaxes exist for expressing a netmask?
- [ ] **ARP**
- [ ] Which layer of the OSI model do ARP belong to?
- [ ] What ARP table includes?
- [ ] Give an example of how ARP protocol works
- [ ] **IP**
- [ ] What it defines?
- [ ] What is a routing table? When is it being used?
- [ ] **TCP**
- [ ] What are the differences between TCP and UDP? When would you prefer to use TCP over UDP?
- [ ] What is a "three-way handshake"?
- [ ] **UDP**
- [ ] Why UDP considered to be unreliable when compared to TCP?
- [ ] Which layer of the OSI model do UDP belong to?
- [ ] **DHCP**
- [ ] How DHCP client locates the DHCP server?
- [ ] Does the client has to be on the same local network as the server?
- [ ] How the exchange between client and the server looks like? what steps are included?
- [ ] Is it using TCP or UDP?
- [ ] What ports is it using?
- [ ] **Netowrk Devices**
- [ ] What is a HUB?
- [ ] What is a Switch?
- [ ] Forwarding table or (aka forwarding information base)
- [ ] What is a Router?
- [ ] What are differences between the three devices above?
- [ ] **Delivery schemes**
- [ ] What is unicast?
- [ ] What is multicast?
- [ ] What is broadcast?
- [ ] What is the broadcast address?
- [ ] What protocols use broadcast?
- [ ] What is a broadcast domain?
- [ ] **VLAN**
- [ ] What are the advantages of using VLAN?
- [ ] What is the range for VLAN IDs?
- [ ] What is a trunk port?
- [ ] **Topologies**
- [ ] Bus topology
- [ ] Ring topology
- [ ] Star topology
- [ ] Tree topology
- [ ] **IP**
- [ ] What layer of OSP model it belongs to?
- [ ] IPv4
- [ ] IPv6
- [ ] Why IPv6 is needed if IPv4 exists?
- [ ] **ICMP**
- [ ] What is it used for?
- [ ] How do you use it?
- [ ] **NAT**
- [ ] What is it and why it's needed?
- [ ] What is SNAT? how it is different from NAT?
- [ ] What is DNAT? how it is different from SNAT?
## Intermediate
- [ ] **Routing**
- [ ] Distance Vector Routing
- [ ] "Count-to-infinity" problem
- [ ] Dijkstra's algorithm
- [ ] **TCP**
- [ ] Silly window syndrome
- [ ] Nagle's algorithm
- [ ] Clark's solution
## Linux Networking
- [ ] **Commands**
- [ ] ping
- [ ] ip addr
- [ ] ip neigh
- [ ] ip link
- [ ] ip route
- [ ] ip netns
- [ ] ethtool
- [ ] netstat
- [ ] traceroute
- [ ] tcpdump
- [ ] mtr
- [ ] **ARP**
- [ ] How to view the contents of the ARP cache?
- [ ] How to initiate an ARP request manually?
- [ ] **MTU**
- [ ] How to see what is the MTU of a given interface?
- [ ] How do you set MTU?
- [ ] **Namespaces**
- [ ] How to create & delete namespaces?
- [ ] How to enter a specific namespace?
- [ ] **Routing*
- [ ] How to display the routing table?
- [ ] How to print the IP address of each router that an IP packet traverses along its path to its destination?
- [ ] **ICMP**
- [ ] How do you use it?
## Academic Level
- [ ] **Digital Modulation**
- [ ] What is Amplitude Shift keying?
- [ ] What is Frequency Shift Keying?
- [ ] What is Phase Shift keying?
- [ ] **Multiplexing**
- [ ] What is it used for and how it works?
- [ ] **Wireless networks**
- [ ] The hidden terminal problem
- [ ] The exposed terminal problem
- [ ] **MACA**
- [ ] What is RTS?
- [ ] What is CTS?
- [ ] **ALOHA**
- [ ] Pure
- [ ] Slotted
- [ ] **Noisy-channel coding theorem**
- [ ] Shannon capacity/limit
- [ ] **CSMA**
- [ ] The Binary Exponential Backoff Algorithm
- [ ] **Stop and Wait**
- [ ] **The adaptive tree walk**
- [ ] **Cryptography**
- [ ] Symmetric
- [ ] Asymmetric
- [ ] Digital Signature
- [ ] **Error Detection**
- [ ] Parity Bit
- [ ] Checksum
- [ ] CRC code
- [ ] Data transmission
- [ ] Piggybacking
- [ ] **Error Correction**
- [ ] Hamming Code
- [ ] Reed Solomon code
- [ ] The NASA binary convolutional code
- [ ] Low-Density Parity Check code
## Cloud Networking (OpenStack Neutron)
- [ ] **Components**
- [ ] API Server
- [ ] Plugins
- [ ] Agents
- [ ] Messaging Queue
- [ ] **Entities**
- [ ] Network
- [ ] Router
- [ ] Port
- [ ] Subnet
- [ ] Security Group
- [ ] **ML2 Plugin**
- [ ] What is it used for?
- [ ] Explain type drivers
- [ ] Explain mechanism drivers
- [ ] **dnsmasq***
- [ ] **Floating IP**
- [ ] **Provider Network**
================================================
FILE: resources/openshift.md
================================================
## OpenShift
### Learn OpenShift
Name | Comments
:------ |:--------:
[Learn OpenShift](https://learn.openshift.com) | Interactive way to learn OpenShift
### CheatSheet
* Login: `oc login -u my_user -p my_password`
#### Projects
* Highlevel overview of the project: `oc status`
* Create a new project: `oc new-project my_project`
* List projects: `oc get projects`
#### Accounts
* The username of the user currently logged in: `oc whoami`
* Add to user "user1" the ability to view the project "wonderland": `oc adm policy add-role-to-user view user1 -n wonderland`
* Add a user as admin to the project "wonderland": `oc adm policy add-role-to-user admin some_user -n wonderland`
* Get a list of all context whihc have ever been created: `oc config get-contexts`
* Check what is the current context: `oc whoami --show-context`
* The OpenShift server currently used: `oc whoami --show-server`
* Get the list of all the OpenShift clusters you have ever logged into: `oc config get-clusters`
#### Pods
* List pods: `oc get po`
* List pods with with node info: `oc get po -o wide`
#### Deployments
* List deployments: `oc get deployments`
#### Service Accounts
* List service accounts: `oc get serviceaccounts`
#### Misc
* Login: `oc login --token= --server=https://:`
================================================
FILE: resources/opensource.md
================================================
## Open Source - Resources
Name | Comments
:------|:------:
[Benevolent dictator for life](https://en.wikipedia.org/wiki/Benevolent_dictator_for_life) |
================================================
FILE: resources/openstack.md
================================================
# OpenStack
## Checklist
- [ ] **Components & Services**
- [ ] Neutron (Networking)
- [ ] Cinder (Storage)
- [ ] Nova (Compute)
- [ ] **Neutron**
- [ ] Agents
- [ ] Server
## Resources
Name | Description
:------|:------:
[OpenStack Operator Tools](https://github.com/openstack/osops-tools-contrib) | Tools and scripts for neutron, nova, etc.
[Infrared](https://github.com/redhat-openstack/infrared) | OpenStack Deployment with Ansible
## Cheatsheet
### Infrared
* Add plugin: `infrared plugin add `
* Add plugin when the spec not in root directory: `infrared plugin add --src-path infrared_plugin`
* Remove plugin: `infrared plugin remove `
* Import workspace (= inventory update): `infrared workspace import https://my-jenkins.com/job/blipblop/1/artifact/workspace.tgz`
* List nodes in inventory/workspace: `infrared workspace node-list`
* If you get error when executing external plugin: "no such role" then you need to create a soft link:
```
mkdir ~/.infrared/plugins//infrared_plugin/roles
ln -s ~/ ~/.infrared/plugins//infrared_plugin/roles/
```
### Servers
* List openstack instances: `openstack server list`
================================================
FILE: resources/packer.md
================================================
## Packer
### Official Resources
* Site: https://www.packer.io
* Docs: https://www.packer.io/docs
### Why Packer
I recommend reading [this discussion](https://www.reddit.com/r/devops/comments/ko3cwq/why_packer) on Reddit
### Blogs Posts
* Getting started with Packer: https://codingshell.com/packer-getting-started
================================================
FILE: resources/prometheus.md
================================================
## Prometheus
### Exporters
Name | Comments
:------|:------:
[Blackbox Exporter](https://github.com/prometheus/blackbox_exporter) | "blackbox probing of endpoints over HTTP, HTTPS, DNS, TCP and ICMP"
Elastic_Search Exporter | Prometheus exporter for various metrics about ElasticSearch, written in Go.
**How to Setup Prometheus in Kubernetes cluster for Elastic Search ?**
To Setup Prometheus in Kubernetes so that we can monitor third party applications like elastic search , we will make use of elastic search exporter which was written in go language and it is now maintained by Prometheus Community. Elastic search exporter will run as a side car container alongside Elastic Search.
now to run as side car container we will just use its image and specify the port:
```
containers:
- name: exporter
image: justwatch/elasticsearch_exporter:1.0.2
ports:
- containerPort: 9108
args: ['-es.uri=http://localhost:9200']
```
Now we will also create a config file for Prometheus:
```
apiVersion: v1
data:
prometheus.yml: |-
global:
scrape_interval: 15s
evaluation_interval: 15s
scrape_configs:
- job_name: "prometheus"
static_configs:
- targets: ["localhost:9090"]
- job_name: "exporter"
static_configs:
- targets: ["elasticsearch:9108"]
kind: ConfigMap
metadata:
name: pr-conf
namespace: elk
```
Here make sure you either write your own namespace or create this elk namespace before running.
Now next we will write a deployment file for Prometheus.
```
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: elk
labels:
app: pr
name: pr
spec:
replicas: 1
selector:
matchLabels:
app: pr
template:
metadata:
creationTimestamp: null
labels:
app: pr
spec:
containers:
- image: prom/prometheus
name: prometheus
ports:
- containerPort: 9090
volumeMounts:
- name: data
mountPath: /etc/prometheus
volumes:
- name: data
configMap:
name: pr-conf
items:
- key: prometheus.yml
path: prometheus.yml
```
Now we will create a service file for nodePort , so that we can access it through our browser through our :
```
apiVersion: v1
kind: Service
metadata:
name: prometheus
namespace: elk
spec:
selector:
app: pr
type: NodePort
ports:
- port: 9090
targetPort: 9090
nodePort: 32200
```
Now we will be able to access the prometheus on 32200 which is our nodePort so start all the services one by one and you're good to go and also we can use Grafana for visualisation too along with prometheus.
================================================
FILE: resources/python.md
================================================
# Python
## Learning Python (Courses)
Name | Description
:------|:------:
[Introduction To Python Programming](https://www.udemy.com/course/pythonforbeginnersintro) | Free, Udemy
[Python for Beginner](https://www.udemy.com/course/python-hackcc) | Free, Udemy
[Learn Python](https://www.youtube.com/watch?v=rfscVS0vtbw&feature=emb_logo) | Free, freeCodeCamp
[Learn Python from Scratch](https://www.educative.io/courses/learn-python-from-scratch) | Free, educative.io
## Learning Python (Tutorials and Interactive platforms)
Name | Description
:------|:------:
[HackerRank](https://www.hackerrank.com) | Python challenges/exercises
[LeetCode](https://leetcode.com) | Python challenges/exercises
[Exercism](https://exercism.io) | Python challenges/exercises
[py4e](https://www.py4e.com) | Python lessons and materials
[W3 Python](https://www.w3schools.com/python) | Python tutorial
[Mode Python Tutorial](https://mode.com/python-tutorial) | Python Tutorial
[Khan Academy](https://www.khanacademy.org) | Python courses and lessons
[Python resources for everybody](https://learnbyexample.github.io/py_resources) | Python written resources
## Modules
Name | Description
:------|:------:
subprocess | Spawn new processes (=execute commands).
agithub | Rest API client for rapid prototyping on any rest API.
## Books
Author | Title | Comments
:------ |:------:|:--------:
Jeff Knupp | Writing Idiomatic Python 3.3 | |
Dane Hillard | [Publishing Python Packages](https://www.manning.com/books/publishing-python-packages) | |
Yong Cui | [Python How-To](https://www.manning.com/books/python-how-to) | |
Pael Anni | [Let's Talk Python](https://www.manning.com/books/lets-talk-python)) | |
## Articles, Tutorials & Blog Posts
Author | Article | Level | Comments
:------ |:------|:--------:|:--------:
Megha Mohan | [Mutable vs Immutable Objects in Python](https://medium.com/@meghamohan/mutable-and-immutable-side-of-python-c2145cf72747) | Beginner | |
Kenneth Reitz | [The Hitchhiker’s Guide to Python](http://docs.python-guide.org/en/latest) | | |
Kenneth Reitz | [Serialization](https://docs.python-guide.org/scenarios/serialization/) | | |
## Libraries, Frameworks & Tools
Name | Description
:------:|:-------:
[Natural Language Toolkit](https://www.nltk.org) | Platform for language processing programming |
[Flask](http://flask.pocoo.org) | Web microframework based on Werkzeug, Jinja 2
[Django](https://www.djangoproject.com) | Web framework with batteries included
[Mypy](http://mypy-lang.org) | Static type checker
[Pandas](https://pandas.pydata.org) | "open source data analysis and manipulation tool"
### Cheat Sheet
#### YAML
* Read YAML
```
with open(r'/file/path') as file:
```
#### Files
* Read remote file(s)
```
from fabric import Connection
with Connection(host) as conn:
with conn.sftp().open(path) as stream:
...
```
#### Dictionaries
* Define dictionary: `some_dict = {'first_number': 2017, 'second_number': 2022}`
* Add item to dictionary: `some_dict['third_number'] = 1991`
* Remove last item: `some_dict.popitem()`
* Remove item by key: `some_dict.pop("third_number")`
* Get all keys without values: `some_dict.keys()`
* Get all values without keys: `some_dict.values()`
* Access item: `some_dict['first_number']` or `some_dict.get('second_number')`
* Number of items in the dictionary: `len(some_dict)`
* Update value of a certain key: `some_dict.update({"first_number": 02017})
## Python Checklist
## Checklist
- [ ] **Data Types**
- [ ] Numbers (int, long, float, complex)
- [ ] List
- [ ] Dictionary
- [ ] String
- [ ] Tuple
- [ ] **Mutability**
- [ ] What data types are mutable?
- [ ] What data types are immutable?
- [ ] **PEP8**
- [ ] What is it?
- [ ] Give an example of three coding conventions Python developers should always follow
- [ ] **Errors & Exceptions**
- [ ] How do you handle exceptions?
- [ ] Iterators
- [ ] What 'enumerate' is used for?
- [ ] **List Comprehensions**
- [ ] Is it better than for loop? If yes, why?
- [ ] How to perform list comprehensions for nested lists?
- [ ] Data serialization
[ ] How you do with Python?
- [ ] **Type Annotations**
- [ ] **Dataclass**
- [ ] **What's that?**
- [ ] What _ is used for in Python?
- [ ] Meta-programming
- [ ] Descriptors
- [ ] Decorators
[ ] Context Managers
- [ ] Buffering Protocol
================================================
FILE: resources/security.md
================================================
## Security
### Books
* [building_secure_and_reliable_systems](https://static.googleusercontent.com/media/sre.google/en//static/pdf/building_secure_and_reliable_systems.pdf)
================================================
FILE: resources/sql.md
================================================
# Linux
## Checklist
Check your Linux educational progress with the following list:
- [ ] **Join**
## Interview Questions
* What is the difference between a hash join to merge join?
* How to compare the performance of inner loop a "when" statement and analytic
function when those are interchangeable?
================================================
FILE: resources/terraform.md
================================================
# Terraform
## Terraform Resources
https://project-awesome.org/shuaibiyy/awesome-terraform
[Terraform Script Builder](https://terraparty.com) | export valid Terraform scripts using UI
## Projects
Name | Comments
:------|:------:
[terraformer](https://github.com/GoogleCloudPlatform/terraformer) | "A CLI tool that generates tf/json and tfstate files based on existing infrastructure (reverse Terraform)."
[terraforming](https://github.com/dtan4/terraforming) | "Export existing AWS resources to Terraform style (tf, tfstate)"
[terrascan](https://github.com/tenable/terrascan) | "Detect compliance and security violations across Infrastructure as Code to mitigate risk before provisioning cloud native infrastructure"
[terraform-kvm](https://github.com/dmacvicar/terraform-provider-libvirt) | "It's a one Terraform Provider for KVM"
### Best Practices
#### tfstate
* Don't edit it manually. tfstate was designed to be manipulated by terraform and not by users directly.
* Store it in secured location (since it can include credentials and sensitive data in general)
* Backup it regularly so you can roll-back easily when needed
* Store it in remote shared storage. This is especially needed when working in a team and the state can be updated by any of the team members
* Enabled versioning if the storage where you store the state file, supports it. Versioning is great for backups and roll-backs in case of an issue.
### Cheat Sheet
* Initialize terraform: `terraform init`
* See what Terraform will execute if running apply: `terraform plan`
* Apply Terraform configuration/code: `terraform apply`
* Cleanup, remove Terraform generated resources: `terraform destroy`
* Dependency graph: `terraform graph`
#### State
* Show state: `terraform show`
* List resources from the state: `terraform state list`
* Rename resource: `terraform state mv`
* Import existing infrastructure into state: `terraform import`
#### Variables
* Pass var: `terraform -var`
* Pass vars file: `terraform -var-file`
* List all outputs: `terraform output`
* Get specific output: `terraform output `
#### Data sources
* Get data: `data...`
#### Backends
* Use variables in a remote backend: `terraform init -backend-config=some_backend_partial_conf.hcl`
#### Workspaces
* Create a new workspace: `terraform workspace new `
* Show current workspace: `terraform workspace show`
#### Productivity
* Console: `terraform console`
================================================
FILE: resources/vault.md
================================================
## Cheat Sheet
* Start Vault server in dev mode (NOT FOR PRODUCTION): `vault server -dev`
* Set up vault client (dev mode):
```
export VAULT_ADDTR='http://'
echo > unseal.key
export VAULT_DEV_ROOT_TOKEN_ID=...
vault status
```
* Write a secret using one pair of key, value: `vault kv put secret/hello foo=bar`
* Write a secret using multiple pairs of key, value: `vault kv put secret/hello foo=bar y=x`
* Get a secret: `vault kv get secret/hello`
================================================
FILE: resources/victoriametrics.md
================================================
## VictoriaMetrics
### Official Resources
* Site: https://victoriametrics.com
* Docs: https://docs.victoriametrics.com
### Why VictoriaMetrics
I recommend to read [case stadies](https://docs.victoriametrics.com/CaseStudies.html) from companies who use VictoriaMetrics in production and [Prominent features](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#prominent-features).
### Blogs Posts
* [Getting started with VictoriaMetrics](https://victoriametrics.com/blog/victoriametrics-monitoring/)
* [How to Choose a Scalable Open Source Time Series Database: The Cost of Scale](https://victoriametrics.com/blog/the-cost-of-scale/)
* [Benchmarking Prometheus-compatible time series databases](https://victoriametrics.com/blog/remote-write-benchmark/)
================================================
FILE: resources/virtual_machines.md
================================================
## Vagrant Resources
Name | Description
:------|:------:
[Official Docs](https://www.vagrantup.com/intro/index.html) | Multi-container applications
## Vagrant Commands
Be familiar with the following commands:
vagrant up
vagrant ssh
================================================
FILE: resources/virtualization.md
================================================
## Virtualization
### Virsh CheatSheet
* List VMs: `virsh list --all`
* Shutdown VMs: `virsh shutdown `
* Delete VMs: `virsh undefine `
* List pools: `virsh pool-list`
* Delete pool content: `virsh pool-delete `
* Deactivate pool: `virsh pool-destroy `
* Delete the pool object: `virsh pool-undefine `
* List networks: `virsh net-list`
* Delete network: `virsh net-undefine && virsh net-destroy `
### Vagrant Resources
Name | Description
:------|:------:
[Official Docs](https://www.vagrantup.com/intro/index.html) | Multi-container applications
#### Vagrant Cheatsheet
* Initialize vagrant file using Fedora image: `vagrant init fedora/33-cloud-base`
* Bring up the VM: `vagrant up`
* SSH into the machine: `vagrant ssh`
* Shutdown: `vagrant halt`
* Delete the VM: `vagrant destroy`
* Reload Vagrant configuration: `vagrant reload`
================================================
FILE: roadmaps.md
================================================
## Roadmaps
### System Design
System Design can be a complex topic. For that reason, I've created a separate repository for you to learn it properly.
