Full Code of flavienbwk/opensearch-docker-compose for AI

main ea3ccdbb762d cached
9 files
30.4 KB
9.5k tokens
1 requests
Download .txt 
Repository: flavienbwk/opensearch-docker-compose
Branch: main
Commit: ea3ccdbb762d
Files: 9
Total size: 30.4 KB

Directory structure:
gitextract_33bkn4cd/

├── .gitignore
├── README.md
├── docker-compose.hot-warm.yml
├── docker-compose.yml
├── generate-certs-hot-warm.sh
├── generate-certs.sh
├── hot-warm-architecture.drawio
├── opensearch-dashboards.yml
└── opensearch.yml

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
certs/
.env

================================================
FILE: README.md
================================================
# OpenSearch - Docker - Compose

![OpenSearch version](https://img.shields.io/badge/OpenSearch%20version-3.4.0-blue)

Dockerized cluster architecture for OpenSearch with compose.

## Key concepts

- OpenSearch is [the successor of OpenDistro](https://opendistro.github.io/for-elasticsearch/blog/2021/06/forward-to-opensearch/)
- OpenSearch = Elasticsearch
- OpenSearch Dashboards = Kibana

> **Note**: Upgrading from 2.x to 3.x requires all index data written with OpenSearch 1.x (or ES 7.x) to be re-indexed into OpenSearch 2.x prior to upgrading.

## Cluster setup

Raise your host's ulimits for ElasticSearch to handle high I/O :

```bash
sudo sysctl -w vm.max_map_count=512000
# Persist this setting in `/etc/sysctl.conf` and execute `sysctl -p`
```

Now, we will generate the certificates for the cluster :

```bash
# You may want to edit the OPENDISTRO_DN variable first
bash generate-certs.sh
```

Start the cluster :

```bash
docker compose up -d
```

Wait about 30 seconds and run `securityadmin` to initialize the security plugin :

```bash
docker compose exec os01 bash -c "chmod +x plugins/opensearch-security/tools/securityadmin.sh && bash plugins/opensearch-security/tools/securityadmin.sh -cd config/opensearch-security -icl -nhnv -cacert config/certificates/ca/ca.pem -cert config/certificates/ca/admin.pem -key config/certificates/ca/admin.key -h localhost"
```

> Find all the configuration files in the container's `/usr/share/opensearch/config/opensearch-security` directory. You might want to [mount them as volumes](https://opendistro.github.io/for-elasticsearch-docs/docs/install/docker-security/).

Access OpenSearch Dashboards through [https://localhost:5601](https://localhost:5601)

Default username is `admin` and password is `admin`

> Take a look at [OpenSearch's internal users documentation](https://opensearch.org/docs/security-plugin/configuration/yaml/) to add, remove or update a user.

## Hot-warm architecture setup

Use a [hot-warm cluster architecture](https://opensearch.org/docs/latest/opensearch/cluster/#advanced-step-7-set-up-a-hot-warm-architecture) if you have data that you rarely want to update or search so you can place them on lower-cost storage nodes.

<center>
    <img alt="Hot-warm architecture schema" src="./hot-warm-architecture.jpg" />
</center>

<details>
<summary>Hot-warm architecture cluster setup instructions...</summary>
<br>

Raise your host's ulimits for ElasticSearch to handle high I/O :

```bash
sudo sysctl -w vm.max_map_count=512000
# Persist this setting in `/etc/sysctl.conf` and execute `sysctl -p`
```

Now, we will generate the certificates for the cluster :

```bash
# You may want to edit the OPENDISTRO_DN variable first
bash generate-certs-hot-warm.sh
```

Adjust `Xms/Xmx` parameters and start the cluster :

```bash
docker compose -f docker-compose.hot-warm.yml up -d
```

Wait about 60 seconds and run `securityadmin` to initialize the security plugin :

```bash
docker compose exec os01 bash -c "chmod +x plugins/opensearch-security/tools/securityadmin.sh && bash plugins/opensearch-security/tools/securityadmin.sh -cd config/opensearch-security -icl -nhnv -cacert config/certificates/ca/ca.pem -cert config/certificates/ca/admin.pem -key config/certificates/ca/admin.key -h localhost"
```

> Find all the configuration files in the container's `/usr/share/opensearch/config/opensearch-security` directory. You might want to [mount them as volumes](https://opendistro.github.io/for-elasticsearch-docs/docs/install/docker-security/).

Access OpenSearch Dashboards through [https://localhost:5601](https://localhost:5601)

Default username is `admin` and password is `admin`

> Take a look at [OpenSearch's internal users documentation](https://opensearch.org/docs/security-plugin/configuration/yaml/) to add, remove or update a user.

</details>

To add an index to a warm node :

```jsn
PUT newindex
{
  "settings": {
    "index.routing.allocation.require.temp": "warm"
  }
}
```

You might want to use [Index State Management (ILM)](https://opensearch.org/docs/latest/im-plugin/index/) to automatically move old indices from _hot_ to _warm_ nodes.

## Why OpenSearch

- Fully open source (including plugins)
- Fully under Apache 2.0 license
- Advanced security plugin (free)
- Alerting plugin (free)
- Allows you to [perform SQL queries against ElasticSearch](https://opendistro.github.io/for-elasticsearch-docs/docs/sql/)
- Maintained by AWS and used for its cloud services


================================================
FILE: docker-compose.hot-warm.yml
================================================
services:

  # Coordinating node (dedicated)
  #   Kind of load-balancer for your cluster. Formerly "client nodes".
  #   Delegates client requests to the shards on the data nodes, 
  #   collects and aggregates the results into one final result, 
  #   and sends this result back to the client.
  # Needs : heavy CPU, medium memory
  os00:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os00
      node.roles: ''
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os00/os00.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os00/os00.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os00/os00.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os00/os00.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data0:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer

  # Master node (dedicated)
  #   Manages the overall operation of a cluster and keeps track of 
  #   the cluster state.
  #   Three dedicated master nodes in three different zones is the 
  #   right approach for almost all production use cases.
  #   3 dedicated master nodes in 3 different zones is the right approach,
  #   Here, we don't do that because we're on 1 machine only.
  #   Master node should not be exposed. Coordinating or ingest nodes can be.
  # Needs : low CPU, low memory
  os01:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os01
      node.roles: 'master'
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os01/os01.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os01/os01.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os01/os01.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os01/os01.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data1:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  

  # Ingest & Data (hot) node
  #   Ingest : Preprocesses data before storing it in the cluster.
  #   Data : Stores and searches data. Performs all data-related 
  #     operations (indexing, searching, aggregating) on local shards.
  #   It is fine to mix both because we're using only 1 server for this cluster.
  #   If you ingest a lot of data, expose a dedicated ingest node.
  # Needs : medium CPU, heavy memory, high-speed storage
  os02:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os02
      node.roles: 'ingest, data'
      node.attr.temp: hot
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os02/os02.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os02/os02.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os02/os02.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os02/os02.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data2:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  # Ingest & Data (hot) node
  #   Ingest : Preprocesses data before storing it in the cluster.
  #   Data : Stores and searches data. Performs all data-related 
  #     operations (indexing, searching, aggregating) on local shards.
  #   It is fine to mix both because we're using only 1 server for this cluster.
  #   If you ingest a lot of data, expose a dedicated ingest node.
  # Needs : medium CPU, heavy memory, high-speed storage
  os03:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os03
      node.roles: 'ingest, data'
      node.attr.temp: hot
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os03/os03.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os03/os03.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os03/os03.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os03/os03.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data3:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  # Ingest & Data (hot) node
  #   Ingest : Preprocesses data before storing it in the cluster.
  #   Data : Stores and searches data. Performs all data-related 
  #     operations (indexing, searching, aggregating) on local shards.
  #   It is fine to mix both because we're using only 1 server for this cluster.
  #   If you ingest a lot of data, expose a dedicated ingest node.
  # Needs : medium CPU, heavy memory, high-speed storage
  os04:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os04
      node.roles: 'ingest, data'
      node.attr.temp: hot
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os04/os04.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os04/os04.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os04/os04.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os04/os04.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data4:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  # Data (warm) node
  #   Ingest : Preprocesses data before storing it in the cluster.
  #   Data : Stores and searches data. Performs all data-related 
  #     operations (indexing, searching, aggregating) on local shards.
  # Needs : lower-speed CPU, heavy memory, lower-speed storage
  os05:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os05
      node.roles: 'data'
      node.attr.temp: warm
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os05/os05.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os05/os05.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os05/os05.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os05/os05.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data5:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  # Data (warm) node
  #   Ingest : Preprocesses data before storing it in the cluster.
  #   Data : Stores and searches data. Performs all data-related 
  #     operations (indexing, searching, aggregating) on local shards.
  # Needs : lower-speed CPU, heavy memory, lower-speed storage
  os06:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os06
      node.roles: 'data'
      node.attr.temp: warm
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os06/os06.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os06/os06.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os06/os06.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os06/os06.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data6:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  # Data (warm) node
  #   Ingest : Preprocesses data before storing it in the cluster.
  #   Data : Stores and searches data. Performs all data-related 
  #     operations (indexing, searching, aggregating) on local shards.
  # Needs : lower-speed CPU, heavy memory, lower-speed storage
  os07:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os07
      node.roles: 'data'
      node.attr.temp: warm
      discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07
      cluster.initial_master_nodes: os01
      plugins.security.ssl.transport.pemkey_filepath: certificates/os07/os07.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os07/os07.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os07/os07.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os07/os07.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data7:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  kibana:
    restart: always
    image: opensearchproject/opensearch-dashboards:3.4.0
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    environment:
      OPENSEARCH_HOSTS: '["https://os00:9200","https://os01:9200","https://os02:9200","https://os03:9200","https://os04:9200","https://os05:9200","https://os06:9200","https://os07:9200"]' # must be a string with no spaces when specified as an environment variable
      DISABLE_INSTALL_DEMO_CONFIG: "true"
    volumes:
      - "./certs:/usr/share/opensearch-dashboards/config/certificates:ro"
      - "./opensearch-dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml"
    ports:
      - 5601:5601

volumes:
  os-data0:
  os-data1:
  os-data2:
  os-data3:
  os-data4:
  os-data5:
  os-data6:
  os-data7:


================================================
FILE: docker-compose.yml
================================================
services:

  os01:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os01
      discovery.seed_hosts: os01,os02,os03
      cluster.initial_master_nodes: os01,os02,os03
      plugins.security.ssl.transport.pemkey_filepath: certificates/os01/os01.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os01/os01.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os01/os01.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os01/os01.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data1:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
  
  os02:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os02
      discovery.seed_hosts: os01,os02,os03
      cluster.initial_master_nodes: os01,os02,os03
      plugins.security.ssl.transport.pemkey_filepath: certificates/os02/os02.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os02/os02.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os02/os02.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os02/os02.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data2:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"
  
  os03:
    restart: always
    image: opensearchproject/opensearch:3.4.0
    environment:
      OPENSEARCH_JAVA_OPTS: "-Xms1024m -Xmx1024m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
      node.name: os03
      discovery.seed_hosts: os01,os02,os03
      cluster.initial_master_nodes: os01,os02,os03
      plugins.security.ssl.transport.pemkey_filepath: certificates/os03/os03.key # relative path
      plugins.security.ssl.transport.pemcert_filepath: certificates/os03/os03.pem
      plugins.security.ssl.http.pemkey_filepath: certificates/os03/os03.key
      plugins.security.ssl.http.pemcert_filepath: certificates/os03/os03.pem
      DISABLE_INSTALL_DEMO_CONFIG: "true"
      JAVA_HOME: /usr/share/opensearch/jdk
      bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping
      network.host: "0.0.0.0"
    ulimits: 
      memlock:
        soft: -1
        hard: -1
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./opensearch.yml:/usr/share/opensearch/config/opensearch.yml"
      - "os-data3:/usr/share/opensearch/data"
      - "./certs:/usr/share/opensearch/config/certificates:ro"

  kibana:
    restart: always
    image: opensearchproject/opensearch-dashboards:3.4.0
    environment:
      OPENSEARCH_HOSTS: '["https://os01:9200","https://os02:9200","https://os03:9200"]' # must be a string with no spaces when specified as an environment variable
      DISABLE_INSTALL_DEMO_CONFIG: "true"
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "1"
    volumes:
      - "./certs:/usr/share/opensearch-dashboards/config/certificates:ro"
      - "./opensearch-dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml"
    ports:
      - 5601:5601

volumes:
  os-data1:
  os-data2:
  os-data3:


================================================
FILE: generate-certs-hot-warm.sh
================================================
#!/bin/bash
# Generate certificates for your OpenSearch cluster

OPENDISTRO_DN="/C=FR/ST=IDF/L=PARIS/O=EXAMPLE"   # Edit here and in opensearch.yml

mkdir -p certs/{ca,os-dashboards}

# Root CA
openssl genrsa -out certs/ca/ca.key 2048
openssl req -new -x509 -sha256 -days 1095 -subj "$OPENDISTRO_DN/CN=CA" -key certs/ca/ca.key -out certs/ca/ca.pem

# Admin
openssl genrsa -out certs/ca/admin-temp.key 2048
openssl pkcs8 -inform PEM -outform PEM -in certs/ca/admin-temp.key -topk8 -nocrypt -v1 PBE-SHA1-3DES -out certs/ca/admin.key
openssl req -new -subj "$OPENDISTRO_DN/CN=ADMIN" -key certs/ca/admin.key -out certs/ca/admin.csr
openssl x509 -req -in certs/ca/admin.csr -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out certs/ca/admin.pem

# OpenSearch Dashboards
openssl genrsa -out certs/os-dashboards/os-dashboards-temp.key 2048
openssl pkcs8 -inform PEM -outform PEM -in certs/os-dashboards/os-dashboards-temp.key -topk8 -nocrypt -v1 PBE-SHA1-3DES -out certs/os-dashboards/os-dashboards.key
openssl req -new -subj "$OPENDISTRO_DN/CN=os-dashboards" -key certs/os-dashboards/os-dashboards.key -out certs/os-dashboards/os-dashboards.csr
openssl x509 -req -in certs/os-dashboards/os-dashboards.csr -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out certs/os-dashboards/os-dashboards.pem
rm certs/os-dashboards/os-dashboards-temp.key certs/os-dashboards/os-dashboards.csr

# Nodes
for NODE_NAME in "os00" "os01" "os02" "os03" "os04" "os05" "os06" "os07"
do
    mkdir "certs/${NODE_NAME}"
    openssl genrsa -out "certs/$NODE_NAME/$NODE_NAME-temp.key" 2048
    openssl pkcs8 -inform PEM -outform PEM -in "certs/$NODE_NAME/$NODE_NAME-temp.key" -topk8 -nocrypt -v1 PBE-SHA1-3DES -out "certs/$NODE_NAME/$NODE_NAME.key"
    openssl req -new -subj "$OPENDISTRO_DN/CN=$NODE_NAME" -key "certs/$NODE_NAME/$NODE_NAME.key" -out "certs/$NODE_NAME/$NODE_NAME.csr"
    openssl x509 -req -extfile <(printf "subjectAltName=DNS:localhost,IP:127.0.0.1,DNS:$NODE_NAME") -in "certs/$NODE_NAME/$NODE_NAME.csr" -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out "certs/$NODE_NAME/$NODE_NAME.pem"
    rm "certs/$NODE_NAME/$NODE_NAME-temp.key" "certs/$NODE_NAME/$NODE_NAME.csr"
done

chmod -R 750 ./certs
chown -R $USER:1000 ./certs


================================================
FILE: generate-certs.sh
================================================
#!/bin/bash
# Generate certificates for your OpenSearch cluster

OPENDISTRO_DN="/C=FR/ST=IDF/L=PARIS/O=EXAMPLE"   # Edit here and in opensearch.yml

mkdir -p certs/{ca,os-dashboards}

# Root CA
openssl genrsa -out certs/ca/ca.key 2048
openssl req -new -x509 -sha256 -days 1095 -subj "$OPENDISTRO_DN/CN=CA" -key certs/ca/ca.key -out certs/ca/ca.pem

# Admin
openssl genrsa -out certs/ca/admin-temp.key 2048
openssl pkcs8 -inform PEM -outform PEM -in certs/ca/admin-temp.key -topk8 -nocrypt -v1 PBE-SHA1-3DES -out certs/ca/admin.key
openssl req -new -subj "$OPENDISTRO_DN/CN=ADMIN" -key certs/ca/admin.key -out certs/ca/admin.csr
openssl x509 -req -in certs/ca/admin.csr -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out certs/ca/admin.pem

# OpenSearch Dashboards
openssl genrsa -out certs/os-dashboards/os-dashboards-temp.key 2048
openssl pkcs8 -inform PEM -outform PEM -in certs/os-dashboards/os-dashboards-temp.key -topk8 -nocrypt -v1 PBE-SHA1-3DES -out certs/os-dashboards/os-dashboards.key
openssl req -new -subj "$OPENDISTRO_DN/CN=os-dashboards" -key certs/os-dashboards/os-dashboards.key -out certs/os-dashboards/os-dashboards.csr
openssl x509 -req -in certs/os-dashboards/os-dashboards.csr -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out certs/os-dashboards/os-dashboards.pem
rm certs/os-dashboards/os-dashboards-temp.key certs/os-dashboards/os-dashboards.csr

# Nodes
for NODE_NAME in "os01" "os02" "os03"
do
    mkdir "certs/${NODE_NAME}"
    openssl genrsa -out "certs/$NODE_NAME/$NODE_NAME-temp.key" 2048
    openssl pkcs8 -inform PEM -outform PEM -in "certs/$NODE_NAME/$NODE_NAME-temp.key" -topk8 -nocrypt -v1 PBE-SHA1-3DES -out "certs/$NODE_NAME/$NODE_NAME.key"
    openssl req -new -subj "$OPENDISTRO_DN/CN=$NODE_NAME" -key "certs/$NODE_NAME/$NODE_NAME.key" -out "certs/$NODE_NAME/$NODE_NAME.csr"
    openssl x509 -req -extfile <(printf "subjectAltName=DNS:localhost,IP:127.0.0.1,DNS:$NODE_NAME") -in "certs/$NODE_NAME/$NODE_NAME.csr" -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out "certs/$NODE_NAME/$NODE_NAME.pem"
    rm "certs/$NODE_NAME/$NODE_NAME-temp.key" "certs/$NODE_NAME/$NODE_NAME.csr"
done

chmod -R 750 ./certs
chown -R $USER:1000 ./certs


================================================
FILE: hot-warm-architecture.drawio
================================================
<mxfile host="Electron" modified="2021-10-08T23:53:09.532Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/15.4.0 Chrome/91.0.4472.164 Electron/13.5.0 Safari/537.36" etag="F2uc_8g4nmpyg63xQ5PZ" version="15.4.0" type="device"><diagram id="qrOvfVEhxfZX0EjswFDh" name="Page-1">7VjJbtswEP0aA+0hgRZLVo6xsx2SIEAOaY+0RElsKFKlqFju15cUqYWWndWuCiQO4HAeOVze44xHmriLrLpkIE9vaATxxLGiauKeTRzHtgNP/JPIWiF+ECggYSjSgzrgHv2BGrQ0WqIIFsZATinmKDfBkBICQ25ggDG6MofFFJur5iCBA+A+BHiIPqCIpwoNPKvDryBK0mZl29I9GWgGa6BIQURXPcg9n7gLRilXraxaQCzJa3hRfhc7etuNMUj4axwu82CZ3EUxv72+XT/MqpMyPTrS6jwBXOoDPwCWTRwfiznnSyZaiWxFgIMhSoTc+nB83TAmVhXiCGO+ShGH9zkIZc9K3A+BpTzDwrJFExS5UixGFRSbnMcI4wXFlNUTuXEM/TAUeMEZfYS9nmh2shQ8u3O9e8g4rHbSYrdki1sKaQY5W4sh2sFtbqi+oK6r9Vr15NZQ2lO6wYC+YEk7c6eBaGgZ3iCJPZDkSlyRAfeIJLAQuNiGDzJJrfqWsTaGWEEIt4u1DLyptyexHM8Uqw2uscRyv8TaKZZn/2diTT99stuMn9GTnf/pJdmMktElaeuVHqWRqIm0SRlPaUIJwOcdOg9L9iQ5rIlltCRRbUmOOodrSnM95BfkfK2rPVByaioDK8R/SPdjT1s/ez1nlZ65NtbaKDhg/FRWfAIglMAGu0Dy/GedXPI0z4slDk9LFsJnWJptF5VBDDh6MuffJpF2vaOI8O4yTJ3p8Yk1nfm2/jYTqLWhuTheArmeY0P2dlPvvwmzQXA2IVjkgBhXxP9dyjp2HqogOZVUJMtvnl3/5onlLaP5vfNoYvkGFByyZgGxX7WG6uyH/l5XPeCBVHLacZxD5awIwCDemrP8MIDL+DA1c/vQM1rNbA8pfTFn7S9N9ZJUl7K2pynBMFv/aCaQRs9Lmp1bbTV+e0xd+gFcJY8DprgPKRoMcs+CUhYhItYmyT8vBTz5tzWs6o/0oIT3cPU5TIkwerg5A3G+nnp25cbRn3qaIsL4Gd/guEhBLpsYkUeTVDNJGkVWiFiIN8usOq29rVx8bWobUt6jtI2JD1ZgTvNmr625HHMKlV4HNddgIt96YaJ3F2/C7N4aquHdu1f3/C8=</diagram></mxfile>

================================================
FILE: opensearch-dashboards.yml
================================================
server.name: os_dashboards
server.host: "0.0.0.0"

opensearch.username: "admin"
opensearch.password: "admin"

# Encrypt traffic between the browser and OpenSearch-Dashboards
server.ssl.enabled: true
server.ssl.certificate: "/usr/share/opensearch-dashboards/config/certificates/os-dashboards/os-dashboards.pem"
server.ssl.key: "/usr/share/opensearch-dashboards/config/certificates/os-dashboards/os-dashboards.key"

# Encrypt traffic between OpenSearch-Dashboards and Opensearch
opensearch.ssl.certificateAuthorities: ["/usr/share/opensearch-dashboards/config/certificates/ca/ca.pem"]
opensearch.ssl.verificationMode: full

# OpenSearch Dashboards 3.x new features
# Enable these for the enhanced Discover experience
data_source.enabled: true
workspace.enabled: true
explore.enabled: true


================================================
FILE: opensearch.yml
================================================
cluster.name: os-cluster
network.host: 0.0.0.0

bootstrap.memory_lock: "true" # along with the memlock settings below, disables swapping

cluster.routing.allocation.disk.threshold_enabled: true
cluster.routing.allocation.disk.watermark.low: 93%
cluster.routing.allocation.disk.watermark.high: 95%

plugins.security.allow_unsafe_democertificates: true
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemtrustedcas_filepath: certificates/ca/ca.pem
plugins.security.ssl.transport.enabled: true
plugins.security.ssl.transport.pemtrustedcas_filepath: certificates/ca/ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false

plugins.security.authcz.admin_dn:
  - 'CN=ADMIN,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
plugins.security.nodes_dn:
  - 'CN=os00,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os01,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os02,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os03,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os04,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os05,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os06,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
  - 'CN=os07,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'
Download .txt 
gitextract_33bkn4cd/

├── .gitignore
├── README.md
├── docker-compose.hot-warm.yml
├── docker-compose.yml
├── generate-certs-hot-warm.sh
├── generate-certs.sh
├── hot-warm-architecture.drawio
├── opensearch-dashboards.yml
└── opensearch.yml
Download .json
Condensed preview — 9 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (33K chars). 
[
  {
    "path": ".gitignore",
    "chars": 11,
    "preview": "certs/\n.env"
  },
  {
    "path": "README.md",
    "chars": 4463,
    "preview": "# OpenSearch - Docker - Compose\n\n![OpenSearch version](https://img.shields.io/badge/OpenSearch%20version-3.4.0-blue)\n\nDo"
  },
  {
    "path": "docker-compose.hot-warm.yml",
    "chars": 14352,
    "preview": "services:\n\n  # Coordinating node (dedicated)\n  #   Kind of load-balancer for your cluster. Formerly \"client nodes\".\n  # "
  },
  {
    "path": "docker-compose.yml",
    "chars": 4492,
    "preview": "services:\n\n  os01:\n    restart: always\n    image: opensearchproject/opensearch:3.4.0\n    environment:\n      OPENSEARCH_J"
  },
  {
    "path": "generate-certs-hot-warm.sh",
    "chars": 2271,
    "preview": "#!/bin/bash\n# Generate certificates for your OpenSearch cluster\n\nOPENDISTRO_DN=\"/C=FR/ST=IDF/L=PARIS/O=EXAMPLE\"   # Edit"
  },
  {
    "path": "generate-certs.sh",
    "chars": 2236,
    "preview": "#!/bin/bash\n# Generate certificates for your OpenSearch cluster\n\nOPENDISTRO_DN=\"/C=FR/ST=IDF/L=PARIS/O=EXAMPLE\"   # Edit"
  },
  {
    "path": "hot-warm-architecture.drawio",
    "chars": 1444,
    "preview": "<mxfile host=\"Electron\" modified=\"2021-10-08T23:53:09.532Z\" agent=\"5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, li"
  },
  {
    "path": "opensearch-dashboards.yml",
    "chars": 787,
    "preview": "server.name: os_dashboards\nserver.host: \"0.0.0.0\"\n\nopensearch.username: \"admin\"\nopensearch.password: \"admin\"\n\n# Encrypt "
  },
  {
    "path": "opensearch.yml",
    "chars": 1114,
    "preview": "cluster.name: os-cluster\nnetwork.host: 0.0.0.0\n\nbootstrap.memory_lock: \"true\" # along with the memlock settings below, d"
  }
]

About this extraction

This page contains the full source code of the flavienbwk/opensearch-docker-compose GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 9 files (30.4 KB), approximately 9.5k tokens. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.

Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.

Extract another repo
Copied to clipboard!