[
{
"path": ".gitignore",
"content": "certs/\n.env"
},
{
"path": "README.md",
"content": "# OpenSearch - Docker - Compose\n\n\n\nDockerized cluster architecture for OpenSearch with compose.\n\n## Key concepts\n\n- OpenSearch is [the successor of OpenDistro](https://opendistro.github.io/for-elasticsearch/blog/2021/06/forward-to-opensearch/)\n- OpenSearch = Elasticsearch\n- OpenSearch Dashboards = Kibana\n\n> **Note**: Upgrading from 2.x to 3.x requires all index data written with OpenSearch 1.x (or ES 7.x) to be re-indexed into OpenSearch 2.x prior to upgrading.\n\n## Cluster setup\n\nRaise your host's ulimits for ElasticSearch to handle high I/O :\n\n```bash\nsudo sysctl -w vm.max_map_count=512000\n# Persist this setting in `/etc/sysctl.conf` and execute `sysctl -p`\n```\n\nNow, we will generate the certificates for the cluster :\n\n```bash\n# You may want to edit the OPENDISTRO_DN variable first\nbash generate-certs.sh\n```\n\nStart the cluster :\n\n```bash\ndocker compose up -d\n```\n\nWait about 30 seconds and run `securityadmin` to initialize the security plugin :\n\n```bash\ndocker compose exec os01 bash -c \"chmod +x plugins/opensearch-security/tools/securityadmin.sh && bash plugins/opensearch-security/tools/securityadmin.sh -cd config/opensearch-security -icl -nhnv -cacert config/certificates/ca/ca.pem -cert config/certificates/ca/admin.pem -key config/certificates/ca/admin.key -h localhost\"\n```\n\n> Find all the configuration files in the container's `/usr/share/opensearch/config/opensearch-security` directory. You might want to [mount them as volumes](https://opendistro.github.io/for-elasticsearch-docs/docs/install/docker-security/).\n\nAccess OpenSearch Dashboards through [https://localhost:5601](https://localhost:5601)\n\nDefault username is `admin` and password is `admin`\n\n> Take a look at [OpenSearch's internal users documentation](https://opensearch.org/docs/security-plugin/configuration/yaml/) to add, remove or update a user.\n\n## Hot-warm architecture setup\n\nUse a [hot-warm cluster architecture](https://opensearch.org/docs/latest/opensearch/cluster/#advanced-step-7-set-up-a-hot-warm-architecture) if you have data that you rarely want to update or search so you can place them on lower-cost storage nodes.\n\n\n ![\"Hot-warm](\"./hot-warm-architecture.jpg\")
\n\n\n\nHot-warm architecture cluster setup instructions...
\n
\n\nRaise your host's ulimits for ElasticSearch to handle high I/O :\n\n```bash\nsudo sysctl -w vm.max_map_count=512000\n# Persist this setting in `/etc/sysctl.conf` and execute `sysctl -p`\n```\n\nNow, we will generate the certificates for the cluster :\n\n```bash\n# You may want to edit the OPENDISTRO_DN variable first\nbash generate-certs-hot-warm.sh\n```\n\nAdjust `Xms/Xmx` parameters and start the cluster :\n\n```bash\ndocker compose -f docker-compose.hot-warm.yml up -d\n```\n\nWait about 60 seconds and run `securityadmin` to initialize the security plugin :\n\n```bash\ndocker compose exec os01 bash -c \"chmod +x plugins/opensearch-security/tools/securityadmin.sh && bash plugins/opensearch-security/tools/securityadmin.sh -cd config/opensearch-security -icl -nhnv -cacert config/certificates/ca/ca.pem -cert config/certificates/ca/admin.pem -key config/certificates/ca/admin.key -h localhost\"\n```\n\n> Find all the configuration files in the container's `/usr/share/opensearch/config/opensearch-security` directory. You might want to [mount them as volumes](https://opendistro.github.io/for-elasticsearch-docs/docs/install/docker-security/).\n\nAccess OpenSearch Dashboards through [https://localhost:5601](https://localhost:5601)\n\nDefault username is `admin` and password is `admin`\n\n> Take a look at [OpenSearch's internal users documentation](https://opensearch.org/docs/security-plugin/configuration/yaml/) to add, remove or update a user.\n\n \n\nTo add an index to a warm node :\n\n```jsn\nPUT newindex\n{\n \"settings\": {\n \"index.routing.allocation.require.temp\": \"warm\"\n }\n}\n```\n\nYou might want to use [Index State Management (ILM)](https://opensearch.org/docs/latest/im-plugin/index/) to automatically move old indices from _hot_ to _warm_ nodes.\n\n## Why OpenSearch\n\n- Fully open source (including plugins)\n- Fully under Apache 2.0 license\n- Advanced security plugin (free)\n- Alerting plugin (free)\n- Allows you to [perform SQL queries against ElasticSearch](https://opendistro.github.io/for-elasticsearch-docs/docs/sql/)\n- Maintained by AWS and used for its cloud services\n"
},
{
"path": "docker-compose.hot-warm.yml",
"content": "services:\n\n # Coordinating node (dedicated)\n # Kind of load-balancer for your cluster. Formerly \"client nodes\".\n # Delegates client requests to the shards on the data nodes, \n # collects and aggregates the results into one final result, \n # and sends this result back to the client.\n # Needs : heavy CPU, medium memory\n os00:\n restart: always\n image: opensearchproject/opensearch:3.4.0\n environment:\n OPENSEARCH_JAVA_OPTS: \"-Xms1024m -Xmx1024m\" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM\n node.name: os00\n node.roles: ''\n discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07\n cluster.initial_master_nodes: os01\n plugins.security.ssl.transport.pemkey_filepath: certificates/os00/os00.key # relative path\n plugins.security.ssl.transport.pemcert_filepath: certificates/os00/os00.pem\n plugins.security.ssl.http.pemkey_filepath: certificates/os00/os00.key\n plugins.security.ssl.http.pemcert_filepath: certificates/os00/os00.pem\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n JAVA_HOME: /usr/share/opensearch/jdk\n bootstrap.memory_lock: \"true\" # along with the memlock settings below, disables swapping\n network.host: \"0.0.0.0\"\n ulimits: \n memlock:\n soft: -1\n hard: -1\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n volumes:\n - \"./opensearch.yml:/usr/share/opensearch/config/opensearch.yml\"\n - \"os-data0:/usr/share/opensearch/data\"\n - \"./certs:/usr/share/opensearch/config/certificates:ro\"\n ports:\n - 9200:9200\n - 9600:9600 # required for Performance Analyzer\n\n # Master node (dedicated)\n # Manages the overall operation of a cluster and keeps track of \n # the cluster state.\n # Three dedicated master nodes in three different zones is the \n # right approach for almost all production use cases.\n # 3 dedicated master nodes in 3 different zones is the right approach,\n # Here, we don't do that because we're on 1 machine only.\n # Master node should not be exposed. Coordinating or ingest nodes can be.\n # Needs : low CPU, low memory\n os01:\n restart: always\n image: opensearchproject/opensearch:3.4.0\n environment:\n OPENSEARCH_JAVA_OPTS: \"-Xms512m -Xmx512m\" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM\n node.name: os01\n node.roles: 'master'\n discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07\n cluster.initial_master_nodes: os01\n plugins.security.ssl.transport.pemkey_filepath: certificates/os01/os01.key # relative path\n plugins.security.ssl.transport.pemcert_filepath: certificates/os01/os01.pem\n plugins.security.ssl.http.pemkey_filepath: certificates/os01/os01.key\n plugins.security.ssl.http.pemcert_filepath: certificates/os01/os01.pem\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n JAVA_HOME: /usr/share/opensearch/jdk\n bootstrap.memory_lock: \"true\" # along with the memlock settings below, disables swapping\n network.host: \"0.0.0.0\"\n ulimits: \n memlock:\n soft: -1\n hard: -1\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n volumes:\n - \"./opensearch.yml:/usr/share/opensearch/config/opensearch.yml\"\n - \"os-data1:/usr/share/opensearch/data\"\n - \"./certs:/usr/share/opensearch/config/certificates:ro\"\n \n\n # Ingest & Data (hot) node\n # Ingest : Preprocesses data before storing it in the cluster.\n # Data : Stores and searches data. Performs all data-related \n # operations (indexing, searching, aggregating) on local shards.\n # It is fine to mix both because we're using only 1 server for this cluster.\n # If you ingest a lot of data, expose a dedicated ingest node.\n # Needs : medium CPU, heavy memory, high-speed storage\n os02:\n restart: always\n image: opensearchproject/opensearch:3.4.0\n environment:\n OPENSEARCH_JAVA_OPTS: \"-Xms1024m -Xmx1024m\" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM\n node.name: os02\n node.roles: 'ingest, data'\n node.attr.temp: hot\n discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07\n cluster.initial_master_nodes: os01\n plugins.security.ssl.transport.pemkey_filepath: certificates/os02/os02.key # relative path\n plugins.security.ssl.transport.pemcert_filepath: certificates/os02/os02.pem\n plugins.security.ssl.http.pemkey_filepath: certificates/os02/os02.key\n plugins.security.ssl.http.pemcert_filepath: certificates/os02/os02.pem\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n JAVA_HOME: /usr/share/opensearch/jdk\n bootstrap.memory_lock: \"true\" # along with the memlock settings below, disables swapping\n network.host: \"0.0.0.0\"\n ulimits: \n memlock:\n soft: -1\n hard: -1\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n volumes:\n - \"./opensearch.yml:/usr/share/opensearch/config/opensearch.yml\"\n - \"os-data2:/usr/share/opensearch/data\"\n - \"./certs:/usr/share/opensearch/config/certificates:ro\"\n \n # Ingest & Data (hot) node\n # Ingest : Preprocesses data before storing it in the cluster.\n # Data : Stores and searches data. Performs all data-related \n # operations (indexing, searching, aggregating) on local shards.\n # It is fine to mix both because we're using only 1 server for this cluster.\n # If you ingest a lot of data, expose a dedicated ingest node.\n # Needs : medium CPU, heavy memory, high-speed storage\n os03:\n restart: always\n image: opensearchproject/opensearch:3.4.0\n environment:\n OPENSEARCH_JAVA_OPTS: \"-Xms1024m -Xmx1024m\" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM\n node.name: os03\n node.roles: 'ingest, data'\n node.attr.temp: hot\n discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07\n cluster.initial_master_nodes: os01\n plugins.security.ssl.transport.pemkey_filepath: certificates/os03/os03.key # relative path\n plugins.security.ssl.transport.pemcert_filepath: certificates/os03/os03.pem\n plugins.security.ssl.http.pemkey_filepath: certificates/os03/os03.key\n plugins.security.ssl.http.pemcert_filepath: certificates/os03/os03.pem\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n JAVA_HOME: /usr/share/opensearch/jdk\n bootstrap.memory_lock: \"true\" # along with the memlock settings below, disables swapping\n network.host: \"0.0.0.0\"\n ulimits: \n memlock:\n soft: -1\n hard: -1\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n volumes:\n - \"./opensearch.yml:/usr/share/opensearch/config/opensearch.yml\"\n - \"os-data3:/usr/share/opensearch/data\"\n - \"./certs:/usr/share/opensearch/config/certificates:ro\"\n \n # Ingest & Data (hot) node\n # Ingest : Preprocesses data before storing it in the cluster.\n # Data : Stores and searches data. Performs all data-related \n # operations (indexing, searching, aggregating) on local shards.\n # It is fine to mix both because we're using only 1 server for this cluster.\n # If you ingest a lot of data, expose a dedicated ingest node.\n # Needs : medium CPU, heavy memory, high-speed storage\n os04:\n restart: always\n image: opensearchproject/opensearch:3.4.0\n environment:\n OPENSEARCH_JAVA_OPTS: \"-Xms1024m -Xmx1024m\" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM\n node.name: os04\n node.roles: 'ingest, data'\n node.attr.temp: hot\n discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07\n cluster.initial_master_nodes: os01\n plugins.security.ssl.transport.pemkey_filepath: certificates/os04/os04.key # relative path\n plugins.security.ssl.transport.pemcert_filepath: certificates/os04/os04.pem\n plugins.security.ssl.http.pemkey_filepath: certificates/os04/os04.key\n plugins.security.ssl.http.pemcert_filepath: certificates/os04/os04.pem\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n JAVA_HOME: /usr/share/opensearch/jdk\n bootstrap.memory_lock: \"true\" # along with the memlock settings below, disables swapping\n network.host: \"0.0.0.0\"\n ulimits: \n memlock:\n soft: -1\n hard: -1\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n volumes:\n - \"./opensearch.yml:/usr/share/opensearch/config/opensearch.yml\"\n - \"os-data4:/usr/share/opensearch/data\"\n - \"./certs:/usr/share/opensearch/config/certificates:ro\"\n \n # Data (warm) node\n # Ingest : Preprocesses data before storing it in the cluster.\n # Data : Stores and searches data. Performs all data-related \n # operations (indexing, searching, aggregating) on local shards.\n # Needs : lower-speed CPU, heavy memory, lower-speed storage\n os05:\n restart: always\n image: opensearchproject/opensearch:3.4.0\n environment:\n OPENSEARCH_JAVA_OPTS: \"-Xms1024m -Xmx1024m\" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM\n node.name: os05\n node.roles: 'data'\n node.attr.temp: warm\n discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07\n cluster.initial_master_nodes: os01\n plugins.security.ssl.transport.pemkey_filepath: certificates/os05/os05.key # relative path\n plugins.security.ssl.transport.pemcert_filepath: certificates/os05/os05.pem\n plugins.security.ssl.http.pemkey_filepath: certificates/os05/os05.key\n plugins.security.ssl.http.pemcert_filepath: certificates/os05/os05.pem\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n JAVA_HOME: /usr/share/opensearch/jdk\n bootstrap.memory_lock: \"true\" # along with the memlock settings below, disables swapping\n network.host: \"0.0.0.0\"\n ulimits: \n memlock:\n soft: -1\n hard: -1\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n volumes:\n - \"./opensearch.yml:/usr/share/opensearch/config/opensearch.yml\"\n - \"os-data5:/usr/share/opensearch/data\"\n - \"./certs:/usr/share/opensearch/config/certificates:ro\"\n \n # Data (warm) node\n # Ingest : Preprocesses data before storing it in the cluster.\n # Data : Stores and searches data. Performs all data-related \n # operations (indexing, searching, aggregating) on local shards.\n # Needs : lower-speed CPU, heavy memory, lower-speed storage\n os06:\n restart: always\n image: opensearchproject/opensearch:3.4.0\n environment:\n OPENSEARCH_JAVA_OPTS: \"-Xms1024m -Xmx1024m\" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM\n node.name: os06\n node.roles: 'data'\n node.attr.temp: warm\n discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07\n cluster.initial_master_nodes: os01\n plugins.security.ssl.transport.pemkey_filepath: certificates/os06/os06.key # relative path\n plugins.security.ssl.transport.pemcert_filepath: certificates/os06/os06.pem\n plugins.security.ssl.http.pemkey_filepath: certificates/os06/os06.key\n plugins.security.ssl.http.pemcert_filepath: certificates/os06/os06.pem\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n JAVA_HOME: /usr/share/opensearch/jdk\n bootstrap.memory_lock: \"true\" # along with the memlock settings below, disables swapping\n network.host: \"0.0.0.0\"\n ulimits: \n memlock:\n soft: -1\n hard: -1\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n volumes:\n - \"./opensearch.yml:/usr/share/opensearch/config/opensearch.yml\"\n - \"os-data6:/usr/share/opensearch/data\"\n - \"./certs:/usr/share/opensearch/config/certificates:ro\"\n \n # Data (warm) node\n # Ingest : Preprocesses data before storing it in the cluster.\n # Data : Stores and searches data. Performs all data-related \n # operations (indexing, searching, aggregating) on local shards.\n # Needs : lower-speed CPU, heavy memory, lower-speed storage\n os07:\n restart: always\n image: opensearchproject/opensearch:3.4.0\n environment:\n OPENSEARCH_JAVA_OPTS: \"-Xms1024m -Xmx1024m\" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM\n node.name: os07\n node.roles: 'data'\n node.attr.temp: warm\n discovery.seed_hosts: os00,os01,os02,os03,os04,os05,os06,os07\n cluster.initial_master_nodes: os01\n plugins.security.ssl.transport.pemkey_filepath: certificates/os07/os07.key # relative path\n plugins.security.ssl.transport.pemcert_filepath: certificates/os07/os07.pem\n plugins.security.ssl.http.pemkey_filepath: certificates/os07/os07.key\n plugins.security.ssl.http.pemcert_filepath: certificates/os07/os07.pem\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n JAVA_HOME: /usr/share/opensearch/jdk\n bootstrap.memory_lock: \"true\" # along with the memlock settings below, disables swapping\n network.host: \"0.0.0.0\"\n ulimits: \n memlock:\n soft: -1\n hard: -1\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n volumes:\n - \"./opensearch.yml:/usr/share/opensearch/config/opensearch.yml\"\n - \"os-data7:/usr/share/opensearch/data\"\n - \"./certs:/usr/share/opensearch/config/certificates:ro\"\n \n kibana:\n restart: always\n image: opensearchproject/opensearch-dashboards:3.4.0\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n environment:\n OPENSEARCH_HOSTS: '[\"https://os00:9200\",\"https://os01:9200\",\"https://os02:9200\",\"https://os03:9200\",\"https://os04:9200\",\"https://os05:9200\",\"https://os06:9200\",\"https://os07:9200\"]' # must be a string with no spaces when specified as an environment variable\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n volumes:\n - \"./certs:/usr/share/opensearch-dashboards/config/certificates:ro\"\n - \"./opensearch-dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml\"\n ports:\n - 5601:5601\n\nvolumes:\n os-data0:\n os-data1:\n os-data2:\n os-data3:\n os-data4:\n os-data5:\n os-data6:\n os-data7:\n"
},
{
"path": "docker-compose.yml",
"content": "services:\n\n os01:\n restart: always\n image: opensearchproject/opensearch:3.4.0\n environment:\n OPENSEARCH_JAVA_OPTS: \"-Xms1024m -Xmx1024m\" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM\n node.name: os01\n discovery.seed_hosts: os01,os02,os03\n cluster.initial_master_nodes: os01,os02,os03\n plugins.security.ssl.transport.pemkey_filepath: certificates/os01/os01.key # relative path\n plugins.security.ssl.transport.pemcert_filepath: certificates/os01/os01.pem\n plugins.security.ssl.http.pemkey_filepath: certificates/os01/os01.key\n plugins.security.ssl.http.pemcert_filepath: certificates/os01/os01.pem\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n JAVA_HOME: /usr/share/opensearch/jdk\n bootstrap.memory_lock: \"true\" # along with the memlock settings below, disables swapping\n network.host: \"0.0.0.0\"\n ulimits: \n memlock:\n soft: -1\n hard: -1\n volumes:\n - \"./opensearch.yml:/usr/share/opensearch/config/opensearch.yml\"\n - \"os-data1:/usr/share/opensearch/data\"\n - \"./certs:/usr/share/opensearch/config/certificates:ro\"\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n ports:\n - 9200:9200\n - 9600:9600 # required for Performance Analyzer\n \n os02:\n restart: always\n image: opensearchproject/opensearch:3.4.0\n environment:\n OPENSEARCH_JAVA_OPTS: \"-Xms1024m -Xmx1024m\" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM\n node.name: os02\n discovery.seed_hosts: os01,os02,os03\n cluster.initial_master_nodes: os01,os02,os03\n plugins.security.ssl.transport.pemkey_filepath: certificates/os02/os02.key # relative path\n plugins.security.ssl.transport.pemcert_filepath: certificates/os02/os02.pem\n plugins.security.ssl.http.pemkey_filepath: certificates/os02/os02.key\n plugins.security.ssl.http.pemcert_filepath: certificates/os02/os02.pem\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n JAVA_HOME: /usr/share/opensearch/jdk\n bootstrap.memory_lock: \"true\" # along with the memlock settings below, disables swapping\n network.host: \"0.0.0.0\"\n ulimits: \n memlock:\n soft: -1\n hard: -1\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n volumes:\n - \"./opensearch.yml:/usr/share/opensearch/config/opensearch.yml\"\n - \"os-data2:/usr/share/opensearch/data\"\n - \"./certs:/usr/share/opensearch/config/certificates:ro\"\n \n os03:\n restart: always\n image: opensearchproject/opensearch:3.4.0\n environment:\n OPENSEARCH_JAVA_OPTS: \"-Xms1024m -Xmx1024m\" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM\n node.name: os03\n discovery.seed_hosts: os01,os02,os03\n cluster.initial_master_nodes: os01,os02,os03\n plugins.security.ssl.transport.pemkey_filepath: certificates/os03/os03.key # relative path\n plugins.security.ssl.transport.pemcert_filepath: certificates/os03/os03.pem\n plugins.security.ssl.http.pemkey_filepath: certificates/os03/os03.key\n plugins.security.ssl.http.pemcert_filepath: certificates/os03/os03.pem\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n JAVA_HOME: /usr/share/opensearch/jdk\n bootstrap.memory_lock: \"true\" # along with the memlock settings below, disables swapping\n network.host: \"0.0.0.0\"\n ulimits: \n memlock:\n soft: -1\n hard: -1\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n volumes:\n - \"./opensearch.yml:/usr/share/opensearch/config/opensearch.yml\"\n - \"os-data3:/usr/share/opensearch/data\"\n - \"./certs:/usr/share/opensearch/config/certificates:ro\"\n\n kibana:\n restart: always\n image: opensearchproject/opensearch-dashboards:3.4.0\n environment:\n OPENSEARCH_HOSTS: '[\"https://os01:9200\",\"https://os02:9200\",\"https://os03:9200\"]' # must be a string with no spaces when specified as an environment variable\n DISABLE_INSTALL_DEMO_CONFIG: \"true\"\n logging:\n driver: \"json-file\"\n options:\n max-size: \"100m\"\n max-file: \"1\"\n volumes:\n - \"./certs:/usr/share/opensearch-dashboards/config/certificates:ro\"\n - \"./opensearch-dashboards.yml:/usr/share/opensearch-dashboards/config/opensearch_dashboards.yml\"\n ports:\n - 5601:5601\n\nvolumes:\n os-data1:\n os-data2:\n os-data3:\n"
},
{
"path": "generate-certs-hot-warm.sh",
"content": "#!/bin/bash\n# Generate certificates for your OpenSearch cluster\n\nOPENDISTRO_DN=\"/C=FR/ST=IDF/L=PARIS/O=EXAMPLE\" # Edit here and in opensearch.yml\n\nmkdir -p certs/{ca,os-dashboards}\n\n# Root CA\nopenssl genrsa -out certs/ca/ca.key 2048\nopenssl req -new -x509 -sha256 -days 1095 -subj \"$OPENDISTRO_DN/CN=CA\" -key certs/ca/ca.key -out certs/ca/ca.pem\n\n# Admin\nopenssl genrsa -out certs/ca/admin-temp.key 2048\nopenssl pkcs8 -inform PEM -outform PEM -in certs/ca/admin-temp.key -topk8 -nocrypt -v1 PBE-SHA1-3DES -out certs/ca/admin.key\nopenssl req -new -subj \"$OPENDISTRO_DN/CN=ADMIN\" -key certs/ca/admin.key -out certs/ca/admin.csr\nopenssl x509 -req -in certs/ca/admin.csr -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out certs/ca/admin.pem\n\n# OpenSearch Dashboards\nopenssl genrsa -out certs/os-dashboards/os-dashboards-temp.key 2048\nopenssl pkcs8 -inform PEM -outform PEM -in certs/os-dashboards/os-dashboards-temp.key -topk8 -nocrypt -v1 PBE-SHA1-3DES -out certs/os-dashboards/os-dashboards.key\nopenssl req -new -subj \"$OPENDISTRO_DN/CN=os-dashboards\" -key certs/os-dashboards/os-dashboards.key -out certs/os-dashboards/os-dashboards.csr\nopenssl x509 -req -in certs/os-dashboards/os-dashboards.csr -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out certs/os-dashboards/os-dashboards.pem\nrm certs/os-dashboards/os-dashboards-temp.key certs/os-dashboards/os-dashboards.csr\n\n# Nodes\nfor NODE_NAME in \"os00\" \"os01\" \"os02\" \"os03\" \"os04\" \"os05\" \"os06\" \"os07\"\ndo\n mkdir \"certs/${NODE_NAME}\"\n openssl genrsa -out \"certs/$NODE_NAME/$NODE_NAME-temp.key\" 2048\n openssl pkcs8 -inform PEM -outform PEM -in \"certs/$NODE_NAME/$NODE_NAME-temp.key\" -topk8 -nocrypt -v1 PBE-SHA1-3DES -out \"certs/$NODE_NAME/$NODE_NAME.key\"\n openssl req -new -subj \"$OPENDISTRO_DN/CN=$NODE_NAME\" -key \"certs/$NODE_NAME/$NODE_NAME.key\" -out \"certs/$NODE_NAME/$NODE_NAME.csr\"\n openssl x509 -req -extfile <(printf \"subjectAltName=DNS:localhost,IP:127.0.0.1,DNS:$NODE_NAME\") -in \"certs/$NODE_NAME/$NODE_NAME.csr\" -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out \"certs/$NODE_NAME/$NODE_NAME.pem\"\n rm \"certs/$NODE_NAME/$NODE_NAME-temp.key\" \"certs/$NODE_NAME/$NODE_NAME.csr\"\ndone\n\nchmod -R 750 ./certs\nchown -R $USER:1000 ./certs\n"
},
{
"path": "generate-certs.sh",
"content": "#!/bin/bash\n# Generate certificates for your OpenSearch cluster\n\nOPENDISTRO_DN=\"/C=FR/ST=IDF/L=PARIS/O=EXAMPLE\" # Edit here and in opensearch.yml\n\nmkdir -p certs/{ca,os-dashboards}\n\n# Root CA\nopenssl genrsa -out certs/ca/ca.key 2048\nopenssl req -new -x509 -sha256 -days 1095 -subj \"$OPENDISTRO_DN/CN=CA\" -key certs/ca/ca.key -out certs/ca/ca.pem\n\n# Admin\nopenssl genrsa -out certs/ca/admin-temp.key 2048\nopenssl pkcs8 -inform PEM -outform PEM -in certs/ca/admin-temp.key -topk8 -nocrypt -v1 PBE-SHA1-3DES -out certs/ca/admin.key\nopenssl req -new -subj \"$OPENDISTRO_DN/CN=ADMIN\" -key certs/ca/admin.key -out certs/ca/admin.csr\nopenssl x509 -req -in certs/ca/admin.csr -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out certs/ca/admin.pem\n\n# OpenSearch Dashboards\nopenssl genrsa -out certs/os-dashboards/os-dashboards-temp.key 2048\nopenssl pkcs8 -inform PEM -outform PEM -in certs/os-dashboards/os-dashboards-temp.key -topk8 -nocrypt -v1 PBE-SHA1-3DES -out certs/os-dashboards/os-dashboards.key\nopenssl req -new -subj \"$OPENDISTRO_DN/CN=os-dashboards\" -key certs/os-dashboards/os-dashboards.key -out certs/os-dashboards/os-dashboards.csr\nopenssl x509 -req -in certs/os-dashboards/os-dashboards.csr -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out certs/os-dashboards/os-dashboards.pem\nrm certs/os-dashboards/os-dashboards-temp.key certs/os-dashboards/os-dashboards.csr\n\n# Nodes\nfor NODE_NAME in \"os01\" \"os02\" \"os03\"\ndo\n mkdir \"certs/${NODE_NAME}\"\n openssl genrsa -out \"certs/$NODE_NAME/$NODE_NAME-temp.key\" 2048\n openssl pkcs8 -inform PEM -outform PEM -in \"certs/$NODE_NAME/$NODE_NAME-temp.key\" -topk8 -nocrypt -v1 PBE-SHA1-3DES -out \"certs/$NODE_NAME/$NODE_NAME.key\"\n openssl req -new -subj \"$OPENDISTRO_DN/CN=$NODE_NAME\" -key \"certs/$NODE_NAME/$NODE_NAME.key\" -out \"certs/$NODE_NAME/$NODE_NAME.csr\"\n openssl x509 -req -extfile <(printf \"subjectAltName=DNS:localhost,IP:127.0.0.1,DNS:$NODE_NAME\") -in \"certs/$NODE_NAME/$NODE_NAME.csr\" -CA certs/ca/ca.pem -CAkey certs/ca/ca.key -CAcreateserial -sha256 -out \"certs/$NODE_NAME/$NODE_NAME.pem\"\n rm \"certs/$NODE_NAME/$NODE_NAME-temp.key\" \"certs/$NODE_NAME/$NODE_NAME.csr\"\ndone\n\nchmod -R 750 ./certs\nchown -R $USER:1000 ./certs\n"
},
{
"path": "hot-warm-architecture.drawio",
"content": "7VjJbtswEP0aA+0hgRZLVo6xsx2SIEAOaY+0RElsKFKlqFju15cUqYWWndWuCiQO4HAeOVze44xHmriLrLpkIE9vaATxxLGiauKeTRzHtgNP/JPIWiF+ECggYSjSgzrgHv2BGrQ0WqIIFsZATinmKDfBkBICQ25ggDG6MofFFJur5iCBA+A+BHiIPqCIpwoNPKvDryBK0mZl29I9GWgGa6BIQURXPcg9n7gLRilXraxaQCzJa3hRfhc7etuNMUj4axwu82CZ3EUxv72+XT/MqpMyPTrS6jwBXOoDPwCWTRwfiznnSyZaiWxFgIMhSoTc+nB83TAmVhXiCGO+ShGH9zkIZc9K3A+BpTzDwrJFExS5UixGFRSbnMcI4wXFlNUTuXEM/TAUeMEZfYS9nmh2shQ8u3O9e8g4rHbSYrdki1sKaQY5W4sh2sFtbqi+oK6r9Vr15NZQ2lO6wYC+YEk7c6eBaGgZ3iCJPZDkSlyRAfeIJLAQuNiGDzJJrfqWsTaGWEEIt4u1DLyptyexHM8Uqw2uscRyv8TaKZZn/2diTT99stuMn9GTnf/pJdmMktElaeuVHqWRqIm0SRlPaUIJwOcdOg9L9iQ5rIlltCRRbUmOOodrSnM95BfkfK2rPVByaioDK8R/SPdjT1s/ez1nlZ65NtbaKDhg/FRWfAIglMAGu0Dy/GedXPI0z4slDk9LFsJnWJptF5VBDDh6MuffJpF2vaOI8O4yTJ3p8Yk1nfm2/jYTqLWhuTheArmeY0P2dlPvvwmzQXA2IVjkgBhXxP9dyjp2HqogOZVUJMtvnl3/5onlLaP5vfNoYvkGFByyZgGxX7WG6uyH/l5XPeCBVHLacZxD5awIwCDemrP8MIDL+DA1c/vQM1rNbA8pfTFn7S9N9ZJUl7K2pynBMFv/aCaQRs9Lmp1bbTV+e0xd+gFcJY8DprgPKRoMcs+CUhYhItYmyT8vBTz5tzWs6o/0oIT3cPU5TIkwerg5A3G+nnp25cbRn3qaIsL4Gd/guEhBLpsYkUeTVDNJGkVWiFiIN8usOq29rVx8bWobUt6jtI2JD1ZgTvNmr625HHMKlV4HNddgIt96YaJ3F2/C7N4aquHdu1f3/C8="
},
{
"path": "opensearch-dashboards.yml",
"content": "server.name: os_dashboards\nserver.host: \"0.0.0.0\"\n\nopensearch.username: \"admin\"\nopensearch.password: \"admin\"\n\n# Encrypt traffic between the browser and OpenSearch-Dashboards\nserver.ssl.enabled: true\nserver.ssl.certificate: \"/usr/share/opensearch-dashboards/config/certificates/os-dashboards/os-dashboards.pem\"\nserver.ssl.key: \"/usr/share/opensearch-dashboards/config/certificates/os-dashboards/os-dashboards.key\"\n\n# Encrypt traffic between OpenSearch-Dashboards and Opensearch\nopensearch.ssl.certificateAuthorities: [\"/usr/share/opensearch-dashboards/config/certificates/ca/ca.pem\"]\nopensearch.ssl.verificationMode: full\n\n# OpenSearch Dashboards 3.x new features\n# Enable these for the enhanced Discover experience\ndata_source.enabled: true\nworkspace.enabled: true\nexplore.enabled: true\n"
},
{
"path": "opensearch.yml",
"content": "cluster.name: os-cluster\nnetwork.host: 0.0.0.0\n\nbootstrap.memory_lock: \"true\" # along with the memlock settings below, disables swapping\n\ncluster.routing.allocation.disk.threshold_enabled: true\ncluster.routing.allocation.disk.watermark.low: 93%\ncluster.routing.allocation.disk.watermark.high: 95%\n\nplugins.security.allow_unsafe_democertificates: true\nplugins.security.ssl.http.enabled: true\nplugins.security.ssl.http.pemtrustedcas_filepath: certificates/ca/ca.pem\nplugins.security.ssl.transport.enabled: true\nplugins.security.ssl.transport.pemtrustedcas_filepath: certificates/ca/ca.pem\nplugins.security.ssl.transport.enforce_hostname_verification: false\n\nplugins.security.authcz.admin_dn:\n - 'CN=ADMIN,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'\nplugins.security.nodes_dn:\n - 'CN=os00,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'\n - 'CN=os01,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'\n - 'CN=os02,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'\n - 'CN=os03,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'\n - 'CN=os04,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'\n - 'CN=os05,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'\n - 'CN=os06,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'\n - 'CN=os07,O=EXAMPLE,L=PARIS,ST=IDF,C=FR'\n"
}
]