Full Code of cortexmedia/Beluga for AI

Repository: cortexmedia/Beluga
Branch: master
Commit: deb3c7174955
Files: 7
Total size: 22.8 KB

Directory structure:
gitextract_6gkmkt0z/

├── README.md
├── bin/
│   └── beluga
├── lib/
│   └── BelugaDeployLib.sh
├── samples/
│   └── BelugaFile
└── scripts/
    ├── BuildAndPush.sh
    ├── Clean.sh
    └── Pull.sh

================================================
FILE CONTENTS
================================================

================================================
FILE: README.md
================================================
![Alt text](/img/logo.png?raw=true "Beluga Docker Deployment Tool")


# **Beluga**
### **Intro**
We've decided to create Beluga in order to fix a very common problem that is the complexity of multi-tenants Docker installations. Indeed, Beluga enables you to quickly draft deployment scripts that will provide you with all the flexibility needed to quickly launch new projects using Docker.

With Beluga, pre-setups are a thing of the past, the kickoff phase becomes extremely short which is an enormous gain if you need to work with multiple clients/projects.

Beluga has no run-time requirements for Linux operating machines and has pretty much no specific requirements outside of actually having Docker installed. Beluga also support private repositories without the need to use Docker Hub.

### **Why Don't You Use Kubernetes or Mesos?**

- Currently Docker doesn't support Multi-Tenant environments
- Therefore, neither Kubernetes and Mesos do from the docker/container layer...
- Many actual issues :
- ```#2918 (PR: #4572) (container root is identical to host root -- volumes can be written and read from as host root inside container).```
- (WIP) Docker doesn't have any ACL. Writing to docker.sock == root.
-    ```#5619 (PR: #6000) (absolute symlinks and symlink path components copy host target).```

### **What Beluga Doesn't Do?**
- Down-to-distro cluster managements
- Docker registry or app management itself
- Magic service discovery
- Infrastructure management

### **How Beluga Works?**
  - Run dockerfiles to build them
  - Push them to repository
  - Pull them when connected to server by ssh
  - That's it!

### **Requirements**
  - A Unix compatible system with RSync
  - SSH
  - Docker Compose
  - Obviously, Docker...
  - Some love

# **Install for OS X Systems using Homebrew**

    brew tap cortexmedia/beluga

### Stable version

     brew install beluga
### Latest version

     brew install --head beluga

#### **How is Beluga Structured**
  - *scripts/* contains all the functions used to build docker containers and deploy them
  - *bin/beluga* Contains the CLI to call the various scripts.
  - *sample/* Example of BelugaFile.
  - *lib/* Common functions used by the various deployment scripts.

#### **Usage**

    beluga [--build args] [--deploy args] [options]
    -b Build the docker container and push to repository.
    -p Connects via ssh to remote host and pulls the images.
    -d Runs the build push and pull options.
    -c Connects via ssh and removed all unused tags and containers.
**Options:**
  ```  -f path/to/BelugaFile Specify the BelugaFile to use.```


##### **Configuration File**

Configuration related to deployment is stored in BelugaFile.

The structure used to grab all the infos related to a container is

    LocalImageName;DockerFilePath;DockerImageName

They are stored in a array like shown in the following example:

    IMAGES_TO_BUILD[0]="mrheaume/sample_project_web;.;sample_project_web"
    IMAGES_TO_BUILD[1]="mrheaume/sample_project_db;DockerPostgres/;sample_project_db"
    IMAGES_TO_BUILD[2]="mrheaume/sample_project_nginx;DockerNginx/;sample_project_nginx”

### **Sample Docker Project with Docker Compose**

    # docker-compose.yml
    web:
        image: my_repository:8080/my-awesome-app
    ports:
        - "5000:5000"
    links:
        - redis
        - nginx
    redis:
        image: my_repository:8080/redis
    nginx:
        image: nginx:latest

### **Contributing**

  The project is currently verified with Shellcheck for bash compatibility (http://www.shellcheck.net/).
  Feel free to ask for a Pull Request if you have awesome ideas for improvements or fixes to bugs.

### **Main Contributors**

  Mathieu Rhéaume <mrheaume@cortexstudio.com>

  **Copyright (c) 2017 [Cortex](http://cortexstudio.com/) ([Mobile Development Studio](http://cortexstudio.com/))**

  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.


================================================
FILE: bin/beluga
================================================
#!/bin/bash
#######################################################################
# Copyright    2015 Cortex Media
# Author    Mathieu Rhéaume <mrheaume@cortex.bz>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#######################################################################

set -e

set_base_directory() {
  BASE_DIR="$(dirname "$(readlink "$0")")"
  DIR="$(cd "$(dirname "$0")" && pwd)"
  FULL_PATH=$(echo "$DIR/$BASE_DIR")
  BASE_DIR="$(cd $FULL_PATH && pwd)"
  export BASE_DIR
}

show_help() {
  echo "Copyright (c) 2015 Cortex (cortex.bz)"
  echo "Beluga (0.0.1-alpha). Usage :"
  echo "beluga [--build args] [--deploy args] [options]"
  echo "-b Build the docker container and push to repository."
  echo "-p Connects via ssh to remote host and pulls the images."
  echo "-d Runs the build push and pull options."
  echo "-c Connects via ssh and removed all unused tags and containers."
  echo "Options:"
  echo "-f path/to/BelugaFile Specify the BelugaFile to use."
}

####################
# MAIN APP RUNTIME #
####################
set_base_directory
for ((i=1;i<=$#;i++))
do
  case ${!i} in
    -f)
      ((i++)) 
      BELUGA_FILE=${!i}
      export BELUGA_FILE
      break
      ;;
  esac
done

for i in "$@"
do
  case $i in
    -b|-b=*|--build|--build=*)
      "$BASE_DIR/../scripts/BuildAndPush.sh"
      exit 0
      shift
      ;;
    -p=*|-p|--pull|--pull=*)
      "$BASE_DIR/../scripts/Pull.sh"
      exit 0
      shift
      ;;
    -c|-c=*|--clean|--clean=*)
      "$BASE_DIR/../scripts/Clean.sh"
      exit 0
      shift
      ;;
    -d=*|-d|--deploy|--deploy=*)
      "$BASE_DIR/../scripts/BuildAndPush.sh"
      if [ $? -ne 0 ]; then
        echo "The build failed."
        exit 1
      else
        "$BASE_DIR/../scripts/Pull.sh"
      fi
      exit 0
      shift
      ;;
    *)
      show_help
      exit 0
      ;;
  esac
done

show_help


================================================
FILE: lib/BelugaDeployLib.sh
================================================
#!/bin/bash
#######################################################################
# Copyright    2015 Cortex Media
# Author    Mathieu Rhéaume <mrheaume@cortex.bz>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#######################################################################
# File : BelugaDeployLib.sh
# Description : Beluga Docker Deployment lib.
# A scavenge of bash functions used to deploy docker containers.
#######################################################################
# This is a list of index references to fit with IMAGES_TO_BUILD structure
# So we don't specify numbers everywhere
#######################################################################
ELOCALIMAGE=0
EDOCKERFILE=1
EDOCKERIMAGENAME=2

################################################
# Function: remove_untagged_containers()       #
# Description : Remove the untagged containers #
################################################
remove_untagged_containers() {
  # Print containers using untagged images: $1 is used with awk's print: 0=line, 1=column 1.
  # NOTE: "[0-9a-f]{12}" does not work with GNU Awk 3.1.7 (RHEL6).
  run_ssh_command "docker ps -a | tail -n +2 | awk '\$2 ~ \"^[0-9a-f]+$\" {print \$1}' | xargs --no-run-if-empty docker rm"
}

################################################
# Function: remove_untagged_images()           #
# Description : Remove the untagged images     #
################################################
remove_untagged_images() {
  # Print untagged images: $1 is used with awk's print: 0=line, 3=column 3.
  # NOTE: intermediate images (via -a) seem to only cause
  # "Error: Conflict, foobarid wasn't deleted" messages.
  # Might be useful sometimes when Docker messed things up?!
  run_ssh_command "docker images | tail -n +2 | awk '\$1 == \"<none>\" {print \$3}' | xargs --no-run-if-empty docker rmi"
}

################################################
# Function: stop_and_build_containers()        #
# Description : This stops the running         #
# containers and tries to build them if needed #
################################################
stop_and_build_containers() {
  run_ssh_command "cd $APP_DIRECTORY; docker-compose -f $DOCKER_COMPOSE_FILE stop; docker-compose -f $DOCKER_COMPOSE_FILE build "
}

################################################
# Function: start_containers_in_background()   #
# Description : This starts the containers in  #
# background.                                  #
################################################
start_containers_in_background() {
  run_ssh_command "cd $APP_DIRECTORY; docker-compose -f $DOCKER_COMPOSE_FILE up -d "
}

################################################
# Function: rotate_containers()                #
# Description : This stops and start the       #
# containers and tries to build them if needed #
################################################
rotate_containers() {
  echo "Rotating docker-compose containers"
  stop_and_build_containers
  start_containers_in_background
}

#############################################
# Function: info_output()                   #
# Description : Output a info text in green #
# background.                               #
#############################################
# Arg 1 : Text to output as info            #
#############################################
info_output() {
  if [ -z "$1" ]
  then
    echo "YOU NEED TO PASS ME A TEXT"
  else
    echo "$(tput setaf 2)INFO: $1 $(tput sgr 0)"
  fi
}

#############################################
# Function: run_ssh_command()               #
# Description : Run a command over ssh with #
# Configuration settings                    #
#############################################
# Arg 1 : Command to run                    #
#############################################
run_ssh_command() {
  if [ -z "$1" ]
  then
    echo "YOU NEED TO SPECIFY A SSH COMMAND"
    exit 1
  else
      for i in "${SERVER_IP[@]}"
      do
        info_output "SSH $i - $1"
        ssh "$DOCKER_USER@$i" $1
      done
  fi
}

##############################################
# Function: pull_docker_image()              #
# Description : Pull a docker container over #
#               ssh on the target machine    #
##############################################
# Arg 1 : Container name                     #
##############################################
pull_docker_image() {
  if [ -z "$1" ]
  then
    echo "YOU NEED TO SPECIFY A CONTAINER NAME"
    exit 1
  else
    run_ssh_command "docker pull $1:latest"
  fi
}

#############################################
# Function: sync_app_files()                #
# Description : Sync files between host and #
#               target machine              #
#############################################
sync_app_files() {
      for i in "${SERVER_IP[@]}"
      do
        info_output "RSYNC $i"
        rsync -r . "$SERVER_USER@$i:$APP_DIRECTORY"
      done
}

##################################################################
# Function: clean_untagged_images()                              #
# Description : Removes all the old untagged docker containers.  #
#               This will skip over the running containers...    #
##################################################################
# ARG 1 : run over ssh on remote hosts                           #
##################################################################
clean_untagged_images() {
  if [ -z "$1" ]
  then
    run_ssh_command "docker rmi $(docker images | grep '<none>' | awk '{print($3)}')"
  else
    docker rmi "$(docker images | grep '<none>' | awk '{print($3)}')"
  fi
}

#################################################
# Function name: build_docker_image
# Description : This function runs a docker build
# on specified docker container name + docker container path
#################################################
# Arg 1 : Container name
# Arg 2 : Recipe folder
#################################################
build_docker_image() {
  if [ -z "$1" ] || [ -z "$2" ]
  then
    echo "You need to specify a container name and receipe folder"
    exit 1
  else
    info_output "Docker build $1 $2"
    docker build -t "$1" "$2"
    if [ $? -ne 0 ]; then
      echo "Docker build failed. Check your stuff mang or ask Chuck Norris"
      exit 1
    else
      echo "Just build $1 with $2"
    fi
  fi
}

#################################################
# Function name: tag_docker_image
# Description : This function runs a docker tags
# a container name with tag name
#################################################
# Arg 1 : Docker container name
# Arg 2 : Docker tag name
#################################################
tag_docker_image() {
  if [ -z "$1" ] || [ -z "$2" ]
  then
    echo "You need to specify a container name and tag name"
    exit 1
  else
    info_output "Docker tag $1 $2"
    docker tag -f "$1" "$2"
    if [ $? -ne 0 ]; then
      echo "Docker tag failed. Check your stuff mang or ask Chuck Norris."
      exit 1
    else
      echo "Just tagged $1 with $2"
    fi
  fi
}

#################################################
# Function name: push_docker_image
# Description : This function pushes the docker images
#################################################
# Arg 1 : Docker repository URL
# Arg 2 : Docker container name
#################################################
push_docker_image() {
  if [ -z "$1" ] || [ -z "$2" ]
  then
    echo "You need to specify a container name and repository"
    exit 1
  else
    info_output "Docker push $1 $2"
    docker push "$1/$2"
    if [ $? -ne 0 ]; then
      echo "Docker push failed. Check your stuff mang or ask Chuck Norris."
      exit 1
    else
      echo "Just pushed $1 to $2"
    fi

  fi
}

#################################################
# Function name: get_config_parameter
# Description : Extracts the selected parameter from string
# Mostly useful for parsing CSV-like strings
#################################################
# Arg 1 : CSV Formatted string
# Arg 2 : Index of item you want
#################################################
get_config_parameter() {
  if [ -z "$1" ] || [ -z "$2" ]
  then
    echo "You need to specify a configuration index and item index"
    exit 1
  else
    imageConfig=$1
    IFS=$ARRAY_DELIMITER read -a parsedconfig <<< "$imageConfig"
    echo "${parsedconfig[$2]}"
  fi
}

#################################################
# Function name: build_images
# Description : Build the docker images
#################################################
build_images() {
  echo "Building images"
  for i in "${IMAGES_TO_BUILD[@]}"
  do
    imageToBuild=$(get_config_parameter "$i" $ELOCALIMAGE)
    dockerFile=$(get_config_parameter "$i" $EDOCKERFILE)
    build_docker_image "$imageToBuild" "$dockerFile"
  done
}

#################################################
# Function name: tag_images
# Description : Tags the docker images
#################################################
tag_images() {
  echo "Tagging images"
  for i in "${IMAGES_TO_BUILD[@]}"
  do
    imageToBuild=$(get_config_parameter "$i" $ELOCALIMAGE)
    dockerFile=$(get_config_parameter "$i" $EDOCKERIMAGENAME)
    dockerTag=$REPOSITORY_URL"/"$dockerFile
    tag_docker_image "$imageToBuild" "$dockerTag"
  done
}

#################################################
# Function name: push_images
# Description : Pushes the docker images to repository
#################################################
push_images() {
  echo "Pushing images"
  for i in "${IMAGES_TO_BUILD[@]}"
  do
    dockerFile=$(get_config_parameter "$i" $EDOCKERIMAGENAME)
    push_docker_image "$REPOSITORY_URL" "$dockerFile"
  done
}

#################################################
# Function name: pull_images
# Description : Pulls the docker images to the target server from private repository
#################################################
pull_images() {
  echo "Pulling images"
  for i in "${IMAGES_TO_BUILD[@]}"
  do
    dockerFile=$(get_config_parameter "$i" $EDOCKERIMAGENAME)
    dockerTag=$REPOSITORY_URL"/"$dockerFile
    pull_docker_image "$dockerTag"
  done
}



================================================
FILE: samples/BelugaFile
================================================
#!/bin/bash
#######################################################################
# Copyright    2015 Cortex Media
# Author    Mathieu Rhéaume <mrheaume@cortex.bz>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#######################################################################
# This config files contains the list of containers that
# BuildDockerImagesAndPushToRepos and PullFromRepository will use
# It keeps references between the docker repos and your own app
# configuration.
#######################################################################
# Docker specific settings.
#######################################################################
REPOSITORY_URL="mydocker.repository.ca:8080"

#######################################################################
# Server specific settings.
#######################################################################
APP_DIRECTORY="/home/ubuntu/sample_app_directory"
DOCKER_COMPOSE_FILE="docker-compose-production.yml"
# User to RSync with
SERVER_USER="ubuntu"
# User that runs docker-compose & containers
DOCKER_USER="ubuntu"

#######################################################################
# List of servers to deploy
#######################################################################
# Single server deployment (OLD Version)
# SERVER_IP="99.999.99.99"
#######################################################################
# Multiple server deployment (New Format)
#######################################################################
declare -a SERVER_IP
SERVER_IP[0]="99.999.99.99"
SERVER_IP[1]="22.222.22.22"

#######################################################################
# Docker specific settings.
# Structure of config line is
# "LocalImageName;DockerFilePath;DockerImageName"
#######################################################################
ARRAY_DELIMITER=";"
declare -a IMAGES_TO_BUILD
IMAGES_TO_BUILD[0]="mrheaume/sample_project_web;.;sample_project_web"
IMAGES_TO_BUILD[1]="mrheaume/sample_project_db;DockerPostgres/;sample_project_db"
IMAGES_TO_BUILD[2]="mrheaume/sample_project_nginx;DockerNginx/;sample_project_nginx"



================================================
FILE: scripts/BuildAndPush.sh
================================================
#!/bin/bash
#######################################################################
# Copyright    2015 Cortex Media
# Author    Mathieu Rhéaume <mrheaume@cortex.bz>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#######################################################################
# File import from config file
#########################################################################
set -e

if [ -z "$BELUGA_FILE" ]; then
  . "./BelugaFile"
else
  . "$BELUGA_FILE"
fi
. "$BASE_DIR/../lib/BelugaDeployLib.sh"

####################
# MAIN APP RUNTIME
####################
build_images
tag_images
push_images



================================================
FILE: scripts/Clean.sh
================================================
#!/bin/bash
#######################################################################
# Copyright    2015 Cortex Media
# Author    Mathieu Rhéaume <mrheaume@cortex.bz>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#########################################################################
# File import from config file
#########################################################################
set -e

if [ -z "$BELUGA_FILE" ]; then
  . "./BelugaFile"
else
  . "$BELUGA_FILE"
fi
. "$BASE_DIR/../lib/BelugaDeployLib.sh"

####################
# MAIN APP RUNTIME #
####################
remove_untagged_containers
remove_untagged_images


================================================
FILE: scripts/Pull.sh
================================================
#!/bin/bash
#######################################################################
# Copyright    2015 Cortex Media
# Author    Mathieu Rhéaume <mrheaume@cortex.bz>
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
###############################################################################
# File import from config file                                                #
###############################################################################
set -e

if [ -z "$BELUGA_FILE" ]; then
  . "./BelugaFile"
else
  . "$BELUGA_FILE"
fi
. "$BASE_DIR/../lib/BelugaDeployLib.sh"

####################
# MAIN APP RUNTIME
####################
sync_app_files
pull_images
rotate_containers

echo "Deployment done"