Reduce the number of shards of an Elasticsearch index (and template)

Written by Claudio Kuenzler - 1 comments

Published on December 27th 2018 - last updated on October 1st 2021 - Listed in ELK Elasticsearch

---

When you run a lot of indexes, this can create quite a large sum of shards in your ELK stack. As the documentation states, the default creates 5 shards per index:

index.number_of_shards
    The number of primary shards that an index should have. Defaults to 5. This setting can only be set at index creation time. It cannot be changed on a closed index. Note: the number of shards are limited to 1024 per index. This limitation is a safety limit to prevent accidental creation of indices that can destabilize a cluster due to resource allocation. The limit can be modified by specifying export ES_JAVA_OPTS="-Des.index.max_number_of_shards=128" system property on every node that is part of the cluster.

Another interesting default setting is the number of replicas of each shard:

 index.number_of_replicas
    The number of replicas each primary shard has. Defaults to 1.

Once a new index was created, the number of shards is fixed. Only by creating a new index the number of shards can be defined; either during the creation of the index itself or by defining the settings in the template. 

Note: Yes, it is possible to change the number of shards on already created indexes, but that means you must re-index that index again, possibly causing downtime.

In my setup, a classical ELK stack, there are a couple of indexes (logstash, filebeat, haproxy, ...)  created every day, typically with the date in the index name (logstash-2018.12.27). By adjusting the shard settings in the templates "logstash" and "filebeat", the indexes created from tomorrow on and later will have a reduced number of shards.

First let's take a backup:

# elk=localhost:9200
# curl $elk/_template/logstash?pretty -u elastic -p > /root/template-logstash.backup

Now create a new file, e.g. /tmp/logstash, based on the backup file.

# cp -p /root/template-logstash.backup /tmp/logstash

Add the "number_of_shards" and "number_of_replicas" into the settings key.

Also make sure, that you remove the "logstash" main key itself. So the file looks like this:

# cat /tmp/logstash
{
    "order" : 0,
    "version" : 60001,
    "index_patterns" : [
      "logstash-*"
    ],
    "settings" : {
      "number_of_shards" : 2,
      "number_of_replicas" : 1,
      "index" : {
        "refresh_interval" : "5s"
      }
    },
    "mappings" : {
[...]
        ],
        "properties" : {
          "@timestamp" : {
            "type" : "date"
          },
          "@version" : {
            "type" : "keyword"
          },
          "geoip" : {
            "dynamic" : true,
            "properties" : {
              "ip" : {
                "type" : "ip"
              },
              "location" : {
                "type" : "geo_point"
              },
              "latitude" : {
                "type" : "half_float"
              },
              "longitude" : {
                "type" : "half_float"
              }
            }
          }
        }
      }
    },
    "aliases" : { }
}

And now this file can be "PUT" into Elasticsearch templates:

# curl -H "Content-Type: application/json" -XPUT $elk/_template/logstash -d "@/tmp/logstash" -u elastic -p
Enter host password for user 'elastic':
{"acknowledged":true}

By checking out the template again, our adjusted shard settings are now showing:

# curl $elk/_template/logstash?pretty -u elastic -p
{
  "logstash" : {
    "order" : 0,
    "version" : 60001,
    "index_patterns" : [
      "logstash-*"
    ],
    "settings" : {
      "index" : {
        "number_of_shards" : "2",
        "number_of_replicas" : "1",
        "refresh_interval" : "5s"
      }
    },
    "mappings" : {

Add a comment

Show form to leave a comment

Comments (newest first)

DenisPat from Canada wrote on Feb 5th, 2020:

You can avoid downtimes with Aliases. I have StuffV1 that has an alias Stuff that points to Stuff*. You copy your index to StuffV2 and erase StuffV1 and still have the alias pointing to your 'stuff' while working on your new index.
Dont hesitate if I aint clear :)

---