18 May 2020

elastic: how many shard should I have

Aim for shard sizes between 10GB and 50GBedit

ref:    https://www.elastic.co/guide/en/elasticsearch/reference/current/size-your-shards.html


To check shard size:
GET _cat/shards?v=true&h=index,prirep,shard,store&s=prirep,store&bytes=gb&index=index_name*
#the store field should be in GB unit




-----------------------old-------------------------

settings": {
    "index": {
      "number_of_shards": "1",
      "number_of_replicas": "1",


number _of_shard: how many shard per index
number_of_replicas: how many clone for each shard



#notes to have different index for each day, might be too much(max shard per ES node is 1000.)
You should try separate the index per month, (and increase number_of_shards).

How much is too many shard? Basically number of shard is ~ number of total CPU core across your cluster.(or you might one to double it if you have Hyper-Threading). More than this, it might not help you to go any quicker in searching data.

#some say it would be ok if your shard size between 2GB - 8GB


ref:  https://qbox.io/blog/optimizing-elasticsearch-how-many-shards-per-index

No comments:

Post a Comment

Terima kasih