In this article, I would show how to build a solid elasticsearch cluster on AWS. In the part 1 we would cover all the details of the AWS EC2 instance creation and preparing the environment to setup the elasticsearch cluster.

First of all to build the AWS cluster we should launch 3 EC2 instances. Why 3, not 2? After this small discussion I found that the cluster of 2 instances is not much better then the cluster of single instance.

As operation system we would use Ubuntu.

As our cluster would not be the real one, then we would use m3.medium kind of instance. If you’re building the real cluster, then, you probably want to have a lot of CPUs (so it would index everything faster) and a lot of RAM (so it can load your big indices to RAM during the search).

On the step 4, it could be reasonable to add some space to your instatnce. As you’re going to index a lot and therefore store a lot of information. In our case we would live it as is, but have in mind that you can do that. Here you can read how to mount the EBS to your instance.

It’s reasonable to add the elasticsearch cluster to the separate security group:

Next steps – reviewing the instances we are going to launch and pointing the key pair to access them.

The instances are launching:

How we should create 3 elastic IPs and assign them to our instances, so we can restart the instances without the fear that they would receive new IP. Also, let’s agree, how we would name our instances, in my case those would be Homer, Marge and Bart.

 

One more thing, to make the configuration even more convinient – we would update our local /etc/hosts to associate the instance names with their IPs.

And now, we would state that hostname on each of the instances (you should do it while you are on the instance).

(the same should be done for the other instances)

After that, we would see the hostname, when we’re logged to the host:

Another thing we should do in the AWS back-end – create the IAM user, so the cluster instances would receive the access to our AWS account and would be able to find each other.

To do that, open the IAM Management Console and create a new user.

You should write down those credentials, because we would use them during the instances configuration. IAM Management Console allows you to download them.

Plus, you should give the user read-only permissions, so it can read the EC2 instances list.

That’s all. Now we are ready to start the cluster configuration.

Elasticsearch cluster on AWS. Part 2 – configuring the elasticsearch.