Repository: flavienbwk/opensearch-docker-compose
Branch: main
Commit: ea3ccdbb762d
Files: 9
Total size: 30.4 KB

Directory structure:
gitextract_33bkn4cd/

├── .gitignore
├── README.md
├── docker-compose.hot-warm.yml
├── docker-compose.yml
├── generate-certs-hot-warm.sh
├── generate-certs.sh
├── hot-warm-architecture.drawio
├── opensearch-dashboards.yml
└── opensearch.yml

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
certs/
.env

================================================
FILE: README.md
================================================
# OpenSearch - Docker - Compose

![OpenSearch version](https://img.shields.io/badge/OpenSearch%20version-3.4.0-blue)

Dockerized cluster architecture for OpenSearch with compose.

## Key concepts

- OpenSearch is [the successor of OpenDistro](https://opendistro.github.io/for-elasticsearch/blog/2021/06/forward-to-opensearch/)
- OpenSearch = Elasticsearch
- OpenSearch Dashboards = Kibana

> **Note**: Upgrading from 2.x to 3.x requires all index data written with OpenSearch 1.x (or ES 7.x) to be re-indexed into OpenSearch 2.x prior to upgrading.

## Cluster setup

Raise your host's ulimits for ElasticSearch to handle high I/O :

```bash
sudo sysctl -w vm.max_map_count=512000
# Persist this setting in `/etc/sysctl.conf` and execute `sysctl -p`
```

Now, we will generate the certificates for the cluster :

```bash
# You may want to edit the OPENDISTRO_DN variable first
bash generate-certs.sh
```

Start the cluster :

```bash
docker compose up -d
```

Wait about 30 seconds and run `securityadmin` to initialize the security plugin :

```bash
docker compose exec os01 bash -c "chmod +x plugins/opensearch-security/tools/securityadmin.sh && bash plugins/opensearch-security/tools/securityadmin.sh -cd config/opensearch-security -icl -nhnv -cacert config/certificates/ca/ca.pem -cert config/certificates/ca/admin.pem -key config/certificates/ca/admin.key -h localhost"
```

> Find all the configuration files in the container's `/usr/share/opensearch/config/opensearch-security` directory. You might want to [mount them as volumes](https://opendistro.github.io/for-elasticsearch-docs/docs/install/docker-security/).

Access OpenSearch Dashboards through [https://localhost:5601](https://localhost:5601)

Default username is `admin` and password is `admin`

> Take a look at [OpenSearch's internal users documentation](https://opensearch.org/docs/security-plugin/configuration/yaml/) to add, remove or update a user.

## Hot-warm architecture setup

Use a [hot-warm cluster architecture](https://opensearch.org/docs/latest/opensearch/cluster/#advanced-step-7-set-up-a-hot-warm-architecture) if you have data that you rarely want to update or search so you can place them on lower-cost storage nodes.

<center>
    <img alt="Hot-warm architecture schema" src="./hot-warm-architecture.jpg" />
</center>

<details>
<summary>Hot-warm architecture cluster setup instructions...</summary>
<br>

Raise your host's ulimits for ElasticSearch to handle high I/O :

```bash
sudo sysctl -w vm.max_map_count=512000
# Persist this setting in `/etc/sysctl.conf` and execute `sysctl -p`
```

Now, we will generate the certificates for the cluster :

```bash
# You may want to edit the OPENDISTRO_DN variable first
bash generate-certs-hot-warm.sh
```

Adjust `Xms/Xmx` parameters and start the cluster :

```bash
docker compose -f docker-compose.hot-warm.yml up -d
```

Wait about 60 seconds and run `securityadmin` to initialize the security plugin :

```bash
docker compose exec os01 bash -c "chmod +x plugins/opensearch-security/tools/securityadmin.sh && bash plugins/opensearch-security/tools/securityadmin.sh -cd config/opensearch-security -icl -nhnv -cacert config/certificates/ca/ca.pem -cert config/certificates/ca/admin.pem -key config/certificates/ca/admin.key -h localhost"
```

> Find all the configuration files in the container's `/usr/share/opensearch/config/opensearch-security` directory. You might want to [mount them as volumes](https://opendistro.github.io/for-elasticsearch-docs/docs/install/docker-security/).

Access OpenSearch Dashboards through [https://localhost:5601](https://localhost:5601)

Default username is `admin` and password is `admin`

> Take a look at [OpenSearch's internal users documentation](https://opensearch.org/docs/security-plugin/configuration/yaml/) to add, remove or update a user.

</details>

To add an index to a warm node :

```jsn
PUT newindex
{
  "settings": {
    "index.routing.allocation.require.temp": "warm"
  }
}
```

You might want to use [Index State Management (ILM)](https://opensearch.org/docs/latest/im-plugin/index/) to automatically move old indices from _hot_ to _warm_ nodes.

## Why OpenSearch

- Fully open source (including plugins)
- Fully under Apache 2.0 license
- Advanced security plugin (free)
- Alerting plugin (free)
- Allows you to [perform SQL queries against ElasticSearch](https://opendistro.github.io/for-elasticsearch-docs/docs/sql/)
- Maintained by AWS and used for its cloud services


================================================
FILE: docker-compose.hot-warm.yml
================================================
services:

  # Coordinating node (dedicated)
  #   Kind of load-balancer for your cluster. Formerly "client nodes".
  #   Delegates client requests to the shards on the data nodes, 
  #   collects and aggregates the results into one final result, 
  #   and sends this result back to the client.
  # Needs : heavy CPU, medium memory
  os00:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os00
      node.roles: ''
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os00/os00.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os00/os00.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os00/os00.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os00/os00.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data0:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer

  # Master node (dedicated)
  #   Manages the overall operation of a cluster and keeps track of 
  #   the cluster state.
  #   Three dedicated master nodes in three different zones is the 
  #   right approach for almost all production use cases.
  #   3 dedicated master nodes in 3 different zones is the right approach,
  #   Here, we don't do that because we're on 1 machine only.
  #   Master node should not be exposed. Coordinating or ingest nodes can be.
  # Needs : low CPU, low memory
  os01:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os01
      node.roles: 'master'
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os01/os01.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os01/os01.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os01/os01.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os01/os01.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data1:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  

  # Ingest & Data (hot) node
  #   Ingest : Preprocesses data before storing it in the cluster.
  #   Data : Stores and searches data. Performs all data-related 
  #     operations (indexing, searching, aggregating) on local shards.
  #   It is fine to mix both because we're using only 1 server for this cluster.
  #   If you ingest a lot of data, expose a dedicated ingest node.
  # Needs : medium CPU, heavy memory, high-speed storage
  os02:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os02
      node.roles: 'ingest, data'
      node.attr.temp: hot
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os02/os02.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os02/os02.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os02/os02.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os02/os02.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data2:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  # Ingest & Data (hot) node
  #   Ingest : Preprocesses data before storing it in the cluster.
  #   Data : Stores and searches data. Performs all data-related 
  #     operations (indexing, searching, aggregating) on local shards.
  #   It is fine to mix both because we're using only 1 server for this cluster.
  #   If you ingest a lot of data, expose a dedicated ingest node.
  # Needs : medium CPU, heavy memory, high-speed storage
  os03:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os03
      node.roles: 'ingest, data'
      node.attr.temp: hot
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os03/os03.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os03/os03.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os03/os03.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os03/os03.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data3:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  # Ingest & Data (hot) node
  #   Ingest : Preprocesses data before storing it in the cluster.
  #   Data : Stores and searches data. Performs all data-related 
  #     operations (indexing, searching, aggregating) on local shards.
  #   It is fine to mix both because we're using only 1 server for this cluster.
  #   If you ingest a lot of data, expose a dedicated ingest node.
  # Needs : medium CPU, heavy memory, high-speed storage
  os04:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os04
      node.roles: 'ingest, data'
      node.attr.temp: hot
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os04/os04.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os04/os04.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os04/os04.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os04/os04.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data4:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  # Data (warm) node
  #   Ingest : Preprocesses data before storing it in the cluster.
  #   Data : Stores and searches data. Performs all data-related 
  #     operations (indexing, searching, aggregating) on local shards.
  # Needs : lower-speed CPU, heavy memory, lower-speed storage
  os05:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os05
      node.roles: 'data'
      node.attr.temp: warm
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os05/os05.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os05/os05.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os05/os05.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os05/os05.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data5:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  # Data (warm) node
  #   Ingest : Preprocesses data before storing it in the cluster.
  #   Data : Stores and searches data. Performs all data-related 
  #     operations (indexing, searching, aggregating) on local shards.
  # Needs : lower-speed CPU, heavy memory, lower-speed storage
  os06:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os06
      node.roles: 'data'
      node.attr.temp: warm
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os06/os06.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os06/os06.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os06/os06.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os06/os06.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data6:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  # Data (warm) node
  #   Ingest : Preprocesses data before storing it in the cluster.
  #   Data : Stores and searches data. Performs all data-related 
  #     operations (indexing, searching, aggregating) on local shards.
  # Needs : lower-speed CPU, heavy memory, lower-speed storage
  os07:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os07
      node.roles: 'data'
      node.attr.temp: warm
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os07/os07.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os07/os07.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os07/os07.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os07/os07.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data7:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  kibana:
    restart: always
    image: opensearchproject/opensearch-dashboards:3.4.0
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    environment:
      OPENSEARCH_HOSTS: '["https://os00:9200","https://os01:9200","https://os02:9200","https://os03:9200","https://os04:9200","https://os05:9200","https://os06:9200","https://os07:9200"]' # must be a string with no spaces when specified as an environment variable
      DISABLE_INSTALL_DEMO_CONFIG: "true"
    volumes:
      - "./certs:/usr/share/opensearch-dashboards/config/certificates:ro"
      - "./opensearch-dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml"
    ports:
      - 5601:5601

volumes:
  os-data0:
  os-data1:
  os-data2:
  os-data3:
  os-data4:
  os-data5:
  os-data6:
  os-data7:


================================================
FILE: docker-compose.yml
================================================
services:

  os01:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os01
      discovery.seed_hosts: os01,os02,os03
      cluster.initial_master_nodes: os01,os02,os03
      plugins.security.ssl.transport.pemkey_filepath: certificates/os01/os01.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os01/os01.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os01/os01.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os01/os01.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data1:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
  
  os02:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os02
      discovery.seed_hosts: os01,os02,os03
      cluster.initial_master_nodes: os01,os02,os03
      plugins.security.ssl.transport.pemkey_filepath: certificates/os02/os02.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os02/os02.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os02/os02.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os02/os02.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data2:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  os03:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os03
      discovery.seed_hosts: os01,os02,os03
      cluster.initial_master_nodes: os01,os02,os03
      plugins.security.ssl.transport.pemkey_filepath: certificates/os03/os03.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os03/os03.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os03/os03.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os03/os03.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data3:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"

  kibana:
    restart: always
    image: opensearchproject/opensearch-dashboards:3.4.0
    environment:
      OPENSEARCH_HOSTS: '["https://os01:9200","https://os02:9200","https://os03:9200"]' # must be a string with no spaces when specified as an environment variable
      DISABLE_INSTALL_DEMO_CONFIG: "true"
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./certs:/usr/share/opensearch-dashboards/config/certificates:ro"
      - "./opensearch-dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml"
    ports:
      - 5601:5601

volumes:
  os-data1:
  os-data2:
  os-data3:


================================================
FILE: generate-certs-hot-warm.sh
================================================
#!/bin/bash
# Generate certificates for your OpenSearch cluster

OPENDISTRO_DN="/C=FR/ST=IDF/L=PARIS/O=EXAMPLE"   # Edit here and in opensearch.yml

mkdir -p certs/{ca,os-dashboards}

# Root CA
openssl genrsa -out certs/ca/ca.key 2048
openssl req -new -x509 -sha256 -days 1095 -subj "$OPENDISTRO_DN/CN=CA" -key certs/ca/ca.key -out certs/ca/ca.pem

# Admin
openssl genrsa -out certs/ca/admin-temp.key 2048
openssl pkcs8 -inform PEM -outform PEM -in certs/ca/admin-temp.key -topk8 -nocrypt -v1 PBE-SHA1-3DES -out certs/ca/admin.key
openssl req -new -subj "$OPENDISTRO_DN/CN=ADMIN" -key certs/ca/admin.key -out certs/ca/admin.csr
openssl x509 -req -in certs/ca/admin.csr -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out certs/ca/admin.pem

# OpenSearch Dashboards
openssl genrsa -out certs/os-dashboards/os-dashboards-temp.key 2048
openssl pkcs8 -inform PEM -outform PEM -in certs/os-dashboards/os-dashboards-temp.key -topk8 -nocrypt -v1 PBE-SHA1-3DES -out certs/os-dashboards/os-dashboards.key
openssl req -new -subj "$OPENDISTRO_DN/CN=os-dashboards" -key certs/os-dashboards/os-dashboards.key -out certs/os-dashboards/os-dashboards.csr
openssl x509 -req -in certs/os-dashboards/os-dashboards.csr -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out certs/os-dashboards/os-dashboards.pem
rm certs/os-dashboards/os-dashboards-temp.key certs/os-dashboards/os-dashboards.csr

# Nodes
for NODE_NAME in "os00" "os01" "os02" "os03" "os04" "os05" "os06" "os07"
do
    mkdir "certs/${NODE_NAME}"
    openssl genrsa -out "certs/$NODE_NAME/$NODE_NAME-temp.key" 2048
    openssl pkcs8 -inform PEM -outform PEM -in "certs/$NODE_NAME/$NODE_NAME-temp.key" -topk8 -nocrypt -v1 PBE-SHA1-3DES -out "certs/$NODE_NAME/$NODE_NAME.key"
    openssl req -new -subj "$OPENDISTRO_DN/CN=$NODE_NAME" -key "certs/$NODE_NAME/$NODE_NAME.key" -out "certs/$NODE_NAME/$NODE_NAME.csr"
    openssl x509 -req -extfile <(printf "subjectAltName=DNS:localhost,IP:127.0.0.1,DNS:$NODE_NAME") -in "certs/$NODE_NAME/$NODE_NAME.csr" -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out "certs/$NODE_NAME/$NODE_NAME.pem"
    rm "certs/$NODE_NAME/$NODE_NAME-temp.key" "certs/$NODE_NAME/$NODE_NAME.csr"
done

chmod -R 750 ./certs
chown -R $USER:1000 ./certs


================================================
FILE: generate-certs.sh
================================================
#!/bin/bash
# Generate certificates for your OpenSearch cluster

OPENDISTRO_DN="/C=FR/ST=IDF/L=PARIS/O=EXAMPLE"   # Edit here and in opensearch.yml

mkdir -p certs/{ca,os-dashboards}

# Root CA
openssl genrsa -out certs/ca/ca.key 2048
openssl req -new -x509 -sha256 -days 1095 -subj "$OPENDISTRO_DN/CN=CA" -key certs/ca/ca.key -out certs/ca/ca.pem

# Admin
openssl genrsa -out certs/ca/admin-temp.key 2048
openssl pkcs8 -inform PEM -outform PEM -in certs/ca/admin-temp.key -topk8 -nocrypt -v1 PBE-SHA1-3DES -out certs/ca/admin.key
openssl req -new -subj "$OPENDISTRO_DN/CN=ADMIN" -key certs/ca/admin.key -out certs/ca/admin.csr
openssl x509 -req -in certs/ca/admin.csr -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out certs/ca/admin.pem

# OpenSearch Dashboards
openssl genrsa -out certs/os-dashboards/os-dashboards-temp.key 2048
openssl pkcs8 -inform PEM -outform PEM -in certs/os-dashboards/os-dashboards-temp.key -topk8 -nocrypt -v1 PBE-SHA1-3DES -out certs/os-dashboards/os-dashboards.key
openssl req -new -subj "$OPENDISTRO_DN/CN=os-dashboards" -key certs/os-dashboards/os-dashboards.key -out certs/os-dashboards/os-dashboards.csr
openssl x509 -req -in certs/os-dashboards/os-dashboards.csr -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out certs/os-dashboards/os-dashboards.pem
rm certs/os-dashboards/os-dashboards-temp.key certs/os-dashboards/os-dashboards.csr

# Nodes
for NODE_NAME in "os01" "os02" "os03"
do
    mkdir "certs/${NODE_NAME}"
    openssl genrsa -out "certs/$NODE_NAME/$NODE_NAME-temp.key" 2048
    openssl pkcs8 -inform PEM -outform PEM -in "certs/$NODE_NAME/$NODE_NAME-temp.key" -topk8 -nocrypt -v1 PBE-SHA1-3DES -out "certs/$NODE_NAME/$NODE_NAME.key"
    openssl req -new -subj "$OPENDISTRO_DN/CN=$NODE_NAME" -key "certs/$NODE_NAME/$NODE_NAME.key" -out "certs/$NODE_NAME/$NODE_NAME.csr"
    openssl x509 -req -extfile <(printf "subjectAltName=DNS:localhost,IP:127.0.0.1,DNS:$NODE_NAME") -in "certs/$NODE_NAME/$NODE_NAME.csr" -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out "certs/$NODE_NAME/$NODE_NAME.pem"
    rm "certs/$NODE_NAME/$NODE_NAME-temp.key" "certs/$NODE_NAME/$NODE_NAME.csr"
done

chmod -R 750 ./certs
chown -R $USER:1000 ./certs


================================================
FILE: hot-warm-architecture.drawio
================================================
<mxfile host="Electron" modified="2021-10-08T23:53:09.532Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/15.4.0 Chrome/91.0.4472.164 Electron/13.5.0 Safari/537.36" etag="F2uc_8g4nmpyg63xQ5PZ" version="15.4.0" type="device"><diagram id="qrOvfVEhxfZX0EjswFDh" name="Page-1">7VjJbtswEP0aA+0hgRZLVo6xsx2SIEAOaY+0RElsKFKlqFju15cUqYWWndWuCiQO4HAeOVze44xHmriLrLpkIE9vaATxxLGiauKeTRzHtgNP/JPIWiF+ECggYSjSgzrgHv2BGrQ0WqIIFsZATinmKDfBkBICQ25ggDG6MofFFJur5iCBA+A+BHiIPqCIpwoNPKvDryBK0mZl29I9GWgGa6BIQURXPcg9n7gLRilXraxaQCzJa3hRfhc7etuNMUj4axwu82CZ3EUxv72+XT/MqpMyPTrS6jwBXOoDPwCWTRwfiznnSyZaiWxFgIMhSoTc+nB83TAmVhXiCGO+ShGH9zkIZc9K3A+BpTzDwrJFExS5UixGFRSbnMcI4wXFlNUTuXEM/TAUeMEZfYS9nmh2shQ8u3O9e8g4rHbSYrdki1sKaQY5W4sh2sFtbqi+oK6r9Vr15NZQ2lO6wYC+YEk7c6eBaGgZ3iCJPZDkSlyRAfeIJLAQuNiGDzJJrfqWsTaGWEEIt4u1DLyptyexHM8Uqw2uscRyv8TaKZZn/2diTT99stuMn9GTnf/pJdmMktElaeuVHqWRqIm0SRlPaUIJwOcdOg9L9iQ5rIlltCRRbUmOOodrSnM95BfkfK2rPVByaioDK8R/SPdjT1s/ez1nlZ65NtbaKDhg/FRWfAIglMAGu0Dy/GedXPI0z4slDk9LFsJnWJptF5VBDDh6MuffJpF2vaOI8O4yTJ3p8Yk1nfm2/jYTqLWhuTheArmeY0P2dlPvvwmzQXA2IVjkgBhXxP9dyjp2HqogOZVUJMtvnl3/5onlLaP5vfNoYvkGFByyZgGxX7WG6uyH/l5XPeCBVHLacZxD5awIwCDemrP8MIDL+DA1c/vQM1rNbA8pfTFn7S9N9ZJUl7K2pynBMFv/aCaQRs9Lmp1bbTV+e0xd+gFcJY8DprgPKRoMcs+CUhYhItYmyT8vBTz5tzWs6o/0oIT3cPU5TIkwerg5A3G+nnp25cbRn3qaIsL4Gd/guEhBLpsYkUeTVDNJGkVWiFiIN8usOq29rVx8bWobUt6jtI2JD1ZgTvNmr625HHMKlV4HNddgIt96YaJ3F2/C7N4aquHdu1f3/C8=</diagram></mxfile>

================================================
FILE: opensearch-dashboards.yml
================================================
server.name: os_dashboards
server.host: "0.0.0.0"

opensearch.username: "admin"
opensearch.password: "admin"

# Encrypt traffic between the browser and OpenSearch-Dashboards
server.ssl.enabled: true
server.ssl.certificate: "/usr/share/opensearch-dashboards/config/certificates/os-dashboards/os-dashboards.pem"
server.ssl.key: "/usr/share/opensearch-dashboards/config/certificates/os-dashboards/os-dashboards.key"

# Encrypt traffic between OpenSearch-Dashboards and Opensearch
opensearch.ssl.certificateAuthorities: ["/usr/share/opensearch-dashboards/config/certificates/ca/ca.pem"]
opensearch.ssl.verificationMode: full

# OpenSearch Dashboards 3.x new features
# Enable these for the enhanced Discover experience
data_source.enabled: true
workspace.enabled: true
explore.enabled: true


================================================
FILE: opensearch.yml
================================================
cluster.name: os-cluster
network.host: 0.0.0.0

bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping

cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 93%
cluster.routing.allocation.disk.watermark.high: 95%

plugins.security.allow_unsafe_democertificates: true
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemtrustedcas_filepath: certificates/ca/ca.pem
plugins.security.ssl.transport.enabled: true
plugins.security.ssl.transport.pemtrustedcas_filepath: certificates/ca/ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false

plugins.security.authcz.admin_dn:
  - 'CN=ADMIN,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
plugins.security.nodes_dn:
  - 'CN=os00,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os01,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os02,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os03,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os04,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os05,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os06,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os07,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'