Getting started with Elastic Search in Docker

Posted on Tuesday, 15th November 2016

Having recently spend a lot of time experimenting with Docker, other than repeatable deployment and runtime environments, one of the great benefits promised by the containerisation movement is how it can supplement your local development environment.

No longer do you need to simultaneously waste time and slow down your local development machine by installing various services like Redis, Postgres and other dependencies. You can simply download a Docker image and boot up your development environment. Then, once you’re finished with it tear it down again.

In fact, a lot of the Docker images for such services are maintained by the development teams and companies themselves.

I’d never fully appreciated this dream until recently when I partook in the quarterly three day hackathon at work, where time was valuable and we didn’t waste time downloading and installing the required JDK just to get Elastic Search installed.

In fact, I was so impressed with Docker and Elastic Search that it compelled me to write this post.

So without further ado, let’s get started.

Elasticsearch is a search server based on Lucene. It provides a distributed, multitenant-capable full-text search engine with a RESTful web interface and schema-free JSON documents.

Now that that’s out of the way, let’s get going. First thing’s first, you’re going to need Docker

Installing Docker

If the title didn’t give it away, we’re going to be setting up Elastic Search up locally using Docker, so if you don’t have Docker installed then you’re going to need to head over to the Docker download page and download/install it.

Next we’re going to need to find the Elastic Search Docker image.

To do this we’re going to head over to Docker Hub and seach for Elastic Search (here’s a direct link for the lazy or those pressed for time).

What is Docker Hub? For new to Docker, Docker Hub is a repository of popular Docker images, many of which are officially owned and supported by the owners of the software.

Pulling and running the Elastic Search Docker image

To run the Elastic Search Docker image we’re first going to need to pull it down to our local machine. To do this open your command prompt or terminal (any directory is fine as nothing is downloaded to the current directory) and execute the following Docker command:

docker pull elasticsearch

Next we want to run our Elastic Search image, do to that we need to type the following command into our terminal:

docker run -d -p 9200:9200 -p 9300:9300 elasticsearch

Let’s breakdown the command:

  • First we’re telling Docker we want to run an image in a container via the ‘run‘ command.

  • The -d argument will run the container in detached mode. This means it will run as a separate background process, as opposed to short-lived process that will run and immediately terminate once it has finished executing.

  • Moving on, the -p arguments tell the container to open and bind our local machine’s port 9200 and 9300 to port 9200 and 9300 in the Docker container.

  • Then at the end we specify the Docker image we wish to start running - in this case, the Elastic Search image.

Note: At this point, if you’re new to Docker then it’s worth knowing that our container’s storage will be deleted when we tear down our Docker container. If you wish to persist the data then we have to use the -v flag to map the Docker’s drive volume to our local disk, as opposed to the default location which is the Docker’s container.

If we want to map the volume to our local disk then we’d need to run the following command instead of the one mentioned above:

docker run -d -v "$HOME/Documents/elasticsearchconf/":/usr/share/elasticsearch/data -p 9200:9200 -p 9300:9300 elasticsearch

This will map our $HOME/Documents/elasticsearchconf folder to the container’s /usr/share/elasticsearch/data directory.

Checking our Elastic Search container is up and running

If the above command worked successfully then we should see the Elastic Search container up and running, we can check this by executing the following command that lists all running containers:

docker ps

To verify Elastic Search is running, you should also be able to navigate to http://localhost:9200 and see output similar to this:

{
  "name" : "f0t5zUn",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "3loHHcMnR_ekDxE1Yc1hpQ",
  "version" : {
    "number" : "5.0.0",
    "build_hash" : "253032b",
    "build_date" : "2016-10-26T05:11:34.737Z",
    "build_snapshot" : false,
    "lucene_version" : "6.2.0"
  },
  "tagline" : "You Know, for Search"
}

Container isn’t running?

If for some reason you container isn’t running, then you can run the following command to see all containers, whether running or not.

docker ps -a

Then once you’ve identified the container you just tried to run (hint: it should be at the top) then run the following command, including the first 3 or 4 characters from the Container Id column:

docker logs 0403

This will print out the containers logs, giving you a bit of information as to what could have gone wrong.

Now that our Docker container is up and running, let’s get our hands dirty with Elastic Search via their RESTful API.

Indexing data

Let’s begin by indexing some data in Elastic Search. We can do this by posting the following product it to our desired index (where product is our index name, and television is our type):

// HTTP POST:
http://localhost:9200/product/television

Message Body:
{"Name": "Samsung Flatscreen Television", "Price": "£899"}

If successful, you should get the following response from Elastic Search:

{
    "_index":"product",
    "_type":"television",
    "_id":"AVhIJ4ACuKcehcHggtFP",
    "_version":1,
    "result":"created",
    "_shards":{
        "total":2,
        "successful":1,
        "failed":0
    },
    "created":true
}

Searching for data

Now we’ve got some data in our index, let’s perform a simple search for it.

Performing the following GET request should return the following data:

// HTTP GET:
http://localhost:9200/_search?q=samsung

The response should look something like this:

{
    "took":16,
    "timed_out":false,
    "_shards":{
        "total":15,
        "successful":15,
        "failed":0
    },
    "hits":{
        "total":1,
        "max_score":0.2876821,
        "hits":[{
                "_index":"product",
                "_type":"television",
                "_id":"AVhIJ4ACuKcehcHggtFP",
                "_score":0.2876821,
                "_source": {
                    "Name": "Samsung Flatscreen Television","Price": "£899"
                }
            }]
        }
}

One of the powerful features of Elastic Search is its  full-text search capabilities, enabling you to perform some truely impressive search queries against your indexed data.

For more on the search options available to you I would recommend you check out this resource.

Deleting Indexed data

To delete indexed data you can perform a delete request, passing the object ID, like so:

// HTTP Delete
http://localhost:9200/product/television/AVhIJ4ACuKcehcHggtFP

Moving Forward

So far we’ve been using Elastic Search’s API query our Elastic Search index. If you’d prefer something more visual that will aid you in your exploration and discovery of the Elastic Search structured query syntax then I’d highly recommend you check out ElasticSearch-Head; a web frontend for your Elastic Search cluster. 

To get started with ElasticSearch-Head you simply clone the repository to your local drive, open the _index.html_ file and point it at your http://localhost:9200 endpoint.

If you have experience issues connecting your web client to your Dockerised Elastic Search cluster then it could be because of CORS permissions. Instead of fiddling around with configurations I simply installed and enabled this Chrome plugin to get around it.

Now you can use the web UI to view the search tab to discover more of Elastic Search’s complex structured query syntax.