Setting up an AWS Environment for a Personal Elasticstack

This is Part 1 of a series of guides about setting up a personal Elasticstack.

1. Building Your Environment in AWS
2. Setting up and Installing Elasticsearch
3. Setting up Kibana
4. Using a Proxy for Kibana with HAProxy
5. Enabling Security and Using Password Authentication
6. Making Kibana Internet Accessible with Cloudfront
7. Securing Cloudfront with Security Groups
8. Inserting Data into Elasticsearch with Logstash

In order to actually build a personal Elasticstack, you are going to want to setup a bit of security and configuration first. I will be using AWS for this as my cloud provider of choice. This could easily be applied to your own servers, or to Azure.

Here is a brief rundown of what instances I will be using throughout this guide:

Role	Instance	Disks	OS
Kibana	t3a.nano	Root 8GB	Centos 7
Proxy	t3a.nano	Root 8GB	Centos 7
Elastic x2	t3a.micro	Root 8GB, Extra 10GB	Centos 7
Logstash	t3a.micro	Root 8GB	Centos 7

I use the region us-east-1 (N. Virginia) in order to keep the costs lower, as where my cluster is doesn’t really matter.

Step 1: Security Groups

First we need to configure the Security Groups to allow the servers to talk to each other. However, we don’t want anything from outside our environment to talk to them. We can use Security Groups to ensure that only wanted communication occurs between instances. (Similar principles apply if you are using Azure instead)

Start by navigating to the Security Groups tab under Network & Security.

Hit the Create security group button on the right.

In here, give it a name and description first (I have called mine ElasticComms). Next leave the Inbound rules blank for now and keep the Outbound Rules as the default All traffic.

Add any tags if you wish and hit Create security group. It should look something like this:

Now click Edit inbound rules and add the following inbound rules:

Type	Port Range	Source	Description
Custom TCP	9200	ElasticComms	Elasticsearch
Custom TCP	9300	ElasticComms	Elastic Communication
Custom TCP	9600	ElasticComms	Logstash
Custom TCP	5601	ElasticComms	Kibana
SSH	22	My IP	Remote Access

In the source, you are going to select the Security Group you are currently editing, in my case ElasticComms. This makes it so only instances with this Security Group can talk to each other over these ports. The SSH is so that you, and only you, can connect to the servers. You can remove this at a later date if you want added security.

It should look something like this:

Now that we have Security Groups, its time to launch our servers.

Step 2: Launch Instances

You are going to probably have to repeat this for each instance you are deploying.

In the Instances tab, hit the Launch instance button.

I am using Centos 7 AMI for this guide, but feel free to use RHEL if you want to pay extra, or Amazon Linux AMI instead.

I am using t3a. instances as they are one of the cheapest instances to run.

I have left everything default, the only change I made was to turn off Unlimited Credits, to save a bit of costs.

For storage, I leave it default, except the Elasticsearch nodes, where I add an extra drive to it. Feel free to make it whatever size you want, but know you can increase the size at any time, but you cannot reduce it. Keep it small if you want to keep it cheap.

In the security groups section, you are going to want to add the security group you made earlier. Hit the Select and existing security group option and ,in my case, select ElasticComms

Once you are happy with everything, launch the instance.

Step 3: Verify Access

The last step is a simple one, try to remote onto your server. If you need help or are unable to do so, checkout the official documentation on how to connect.

Now that are are done setting up and you can connect, lets move onto Part 2: Setting up and Installing Elasticsearch.

---

Any thoughts, concerns, mistakes? Let me know in the comments or via the Contact page.