In this tutorial, we will provide step-by-step instructions on how to encrypt communication among Elasticsearch, Kibana, and Logstash. After the setup, Elasticsearch and Kibana can be accessed by web browsers via https.

Before we start, make sure the following conditions are satisfied or agreed:

  • Elastic services are running with no issues on the server.
  • The directory layout for the Elastic stack is known. We use the directory layout of RPM in this tutorial. For further information, check this website: Directory layout of RPM
  • If shell prompt asks for additional options, press ENTER to choose the default one.
  • Using sudo before any command if Permission denied occurs
  • If any service fails to start, check the corresponding log. If the log indicates access denied for some files, make sure you have given the access permission for the files. Check this website for further information.
  • Some steps like creating directories, copying files, and editing files may be omitted.
  • Read the comments for the commands carefully. Make changes for the commands according to your situation.
  • Since we use self-sign CA in this tutorial, the web browser will show the certificate is not trusted in the case. It is ok, later I will write a tutorial on how to use third-party CA to make sure the certificate is trusted by the web browser. Please follow me to receive the latest notification about my articles.

Step 1: Using self-sign CA generate from Elasticsearch:

<span style="font-family: terminal, monaco;"><strong>cd</strong></span> to the binary scripts directory. In my case, cd /usr/share/elasticsearch. Make sure you type your own domain name or IP address after CN= in the following commands

bin/elasticsearch-certutil ca --ca-dn CN=your_domain_name

bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 -name "CN=your_domain_name,OU=Consulting Team,DC=your_domain_name,DC=com"

At this point, you have obtained two files: elastic-certificates.p12 and elastic-stack-ca.p12. Now we are going to set up passwords for built-in users. Make sure the passwords are recorded and protected in a safe place. Once the password is set up, it won’t allow being recovered. Use bin/elasticsearch-setup-passwords auto to let the script generate the passwords or use /bin/elasticsearch-setup-passwords interactive to customize the passwords for each build-in users by yourself.

Use the following command to create a directory certs under the conf directory and copy elastic-certificates.p12 to this directory.

mkdir -p /etc/elasticsearch/certs

cp /usr/share/elasticsearch/elastic-certificates.p12 /etc/elasticsearch/certs

Add the following lines in the Elasticsearch configuration file. In my case, the path is /etc/elasticsearch/elasticsearch.yml .

http.host: 0.0.0.0 # accept request from remote

xpack.security.enabled: true

xpack.security.transport.ssl.enabled: true

xpack.security.transport.ssl.verification_mode: certificate

xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12

xpack.security.transport.ssl.truststore.path: certs/elastic-certificates.p12

xpack.security.http.ssl.enabled: true

xpack.security.http.ssl.keystore.path: certs/elastic-certificates.p12

xpack.security.http.ssl.truststore.path: certs/elastic-certificates.p12

xpack.security.http.ssl.client_authentication: optional

Restart Elasticsearch service to make sure the above changes take effect and no error occurred at this point.

Step 2: Enable cross-origin resource sharing for Elasticsearch

Enable CORS so that a browser on another origin can send requests to Elasticsearch. Add the following lines in the Elasticsearch configuration file. In my case, the path is /etc/elasticsearch/elasticsearch.yml .

Attention, for the setting http.cors.allow-origin , it should be the URL of the website which you are giving permission to connect. In my case, I will use “https://ai.query.ai” .

http.cors.enabled: true

http.cors.allow-origin: “https://ai.query.ai”

http.cors.allow-methods : OPTIONS, HEAD, GET, POST, PUT, DELETE

http.cors.allow-headers : Authorization, X-Requested-With,X-Auth-Token,Content-Type, Content-Length

Step 3: Encrypt communication between Kibana and web browser:

cd to certs folder under the Elasticsearch conf directory. Run the following commands to obtain the key file and certificate file from elastic-certificates.p12 .

openssl pkcs12 -in /elastic-certificates.p12 -out newfile.crt.pem -clcerts -nokeys

penssl pkcs12 -in ./elastic-certificates.p12 -out newfile.key.pem -nocerts -nodes

At this point two new files are obtained: newfile.crt.pem and newfile.key.pem . Create a directory named certs under Kibana conf folder, and copy these two files to certs directory. Add the following lines to the Kibana configuration file to enable HTTP SSL for Kibana. In my case, it is located at /etc/kibana/kibana.yml

server.host: 0.0.0.0

server.ssl.enabled: true

server.ssl.key: /etc/kibana/certs/newfile.key.pem

server.ssl.certificate: /etc/kibana/certs/newfile.crt.pem

Step 4: Encrypt communication between Kibana and Elasticsearch:

See step1, we’ve obtained the password for user Kibana. Add these lines to the Kibana configuration file to enable TLS between Elasticsearch and Kibana:

elasticsearch.hosts: [“https://localhost:9200″] # https, not http

elasticsearch.username: “kibana”

elasticsearch.password: “XXXXXXXXX” # the password for user kibana

elasticsearch.ssl.verificationMode: none

Restart Kibana service, access the Kibana UI on your web browser. Make sure the Kibana UI shows up and is connected by https.

Step 5: Encrypt communication between Logstash and Elasticsearch:

There are configurations related to Elasticsaerch in Logstash’s pipeline. Since we’ve enabled https and built-in users for Elasticsearch. We should do the following changes in every pipeline config file in order to make the connection to Elasticsearch via https.

First, create a directory called certs under Logstash conf directory. In my case, it is located at /etc/logstash . Then, copy newfile.crt.pem to the certs directory.

Make changes on Elasticsearch output plugin for every Logstash pipeline

output {

elasticsearch {

hosts => “https://localhost:9200″ #make sure it’s https

index => “nginx-%{+YYYY.MM.dd}”

document_type => “nginx_logs”

user => logstash_system

password => xxxxxxxxx # obtained by step1

cacert => ‘/etc/logstash/certs/newfile.crt.pem’ # generated from step3

ssl_certificate_verification => false

}

}

Restart Logstash in order for the changes to take effect.

All Done:

After the above changes, the communications among Elasticsearch, Kibana, and Logstash are encrypted. 

Interested in similar content? Follow our linkedin page!