Elasticsearch Garbage Collector

If you're using Elasticsearch you'll sooner or later have to deal with diskspace issues. The setup I currently have to manage gathers 200 to 300 million docs per 24 hours and a solution was needed to always guarantee enough free diskspace so that Elasticsearch wouldn't fail.

The following bash script is the last incarnation of this quest to have an automatic “ringbuffer”:

#!/bin/bash

  LOCKFILE=/var/run/egc.lock

# Check if Lockfile exists

  if [ -e ${LOCKFILE} ] && kill -0 `cat ${LOCKFILE}`; then
    echo "EGC process already running"
    exit 1
  fi

# Make sure the Lockfile is removed 
# when we exit and then claim it

  trap "rm -f ${LOCKFILE}; exit" INT TERM EXIT

# Create Lockfile

  echo $$ > ${LOCKFILE}

# Always keep a minimum of 30GB free in logdata
# by sacrificing oldest index (ringbuffer)

  DF=$(/bin/df /dev/md0 | sed '1d' | awk '{print $4}')

  if [ ${DF} -le 30000000 ]; then
    INDEX=$(/bin/ls -1td /logdata/dntx-es/nodes/0/indices/logstash-* | tail -1 | xargs -n 1 basename)
    curl -XDELETE "http://localhost:9200/${INDEX}"
  fi

# Check & clean elasticsearch logs 
# if disk usage is > 10GB

  DU=$(/usr/bin/du /var/log/elasticsearch/ | awk '{print $1}')

  if [ ${DU} -ge 10000000 ]; then
    rm /var/log/elasticsearch/elasticsearch.log.20*
  fi

# Remove Lockfile

  rm -f ${LOCKFILE}

  exit 0

Make sure to check/modify the script to reflect your particular setup: It's very likely that your paths and device names are different.

It runs every 10 minutes (as a cron job) and checks the available space on the device where Elasticsearch stores its indices. In this example /dev/md0 is mounted on /logdata. If md0 has less than 30GB of free diskspace it automagically finds the oldest Elasticsearch index and drops it via Elasticsearch's REST API without service interruption (no stop/restart of Elasticsearch required).

A simple locking mechanism will prevent multiple running instances in case of timing issues. All you need is curl for it to work and it will increase your storage efficiency so that you can always have as much past data available as your storage allows without the risk of full disk issues or the hassle of manual monitoring & maintaining.

devops, big-data, monitoring, elasticsearch, software

Discussion

fe80:c955:24f1:7d93:7d82:2c7d:278e:fbc7, 2013/12/04 09:22

Really good post

chrono, 2014/02/20 15:06

The script has been updated to only find logstash-* indexes, otherwise it could tamper with river or marvel/kibana indexes.

Leo, 2023/11/29 07:05

If you are interested in your favorite actors' heights and detailed information, please take a look at celeb height wiki database.

OKBet, 2023/12/14 02:20

It's nice to see your writing, which is exactly what I need, it's very detailed 15% First Deposit Bonus ng OKBet

colorgame, 2024/01/29 03:28

Salamat sa pagbabahagi ng iyong kaalaman. Ito'y naging inspirasyon sa akin. Sana'y masilip mo ang aking website:

christmas jewelry, 2024/01/29 03:52

I deeply appreciate your article, it provided much-needed guidance. Please consider visiting my website:

25fox, 2024/02/12 23:40

Buy a driver's license Hello, welcome to the world's largest online driver's license organization. We sell authentic and registered driving licenses and we have several driving schools with which we collaborate.

https://origineelrijbewijskopen.com/

https://origineelrijbewijskopen.com/2023/11/10/rijbewijs-kopen-telegram/

https://origineelrijbewijskopen.com/2023/10/24/rijbewijs-kopen-nederland/ https://origineelrijbewijskopen.com/2023/11/01/rijbewijs-kopen-belgie/

https://origineelrijbewijskopen.com/2023/11/07/rijbewijs-a-kopen/

https://origineelrijbewijskopen.com/2023/11/07/rijbewijs-kopen-prijs/

https://origineelrijbewijskopen.com/2023/11/07/rijbewijs-te-koop-belgie/

https://origineelrijbewijskopen.com/2023/11/10/belgisch-rijbewijs-kopen/

https://origineelrijbewijskopen.com/2023/11/10/rijbewijs-kopen-duitsland/ https://origineelrijbewijskopen.com/2023/11/10/rijbewijs-examens-kopen/

https://origineelrijbewijskopen.com/2023/11/10/rijbewijs-kopen-hongarije/

What a nice post! I'm so happy to read this. What you wrote was very helpful to me. Thank you. Actually, I run a site similar to yours. If you have time, could you visit my site? Please leave your comments after reading what I wrote. If you do so, I will actively reflect your opinion. I think it will be a great help to run my site. Have a good day.

https://cf-cc-qualitylab.com/

https://cf-cc-qualitylab.com/product/american-dollars-for-sale/

https://cf-cc-qualitylab.com/product/buy-counterfeit-aud-online/

https://cf-cc-qualitylab.com/product/buy-fake-british-pounds-online/

https://cf-cc-qualitylab.com/product/buy-fake-euro-banknotes-online/

https://cf-cc-qualitylab.com/product/buy-new-zealand-dollars-online/ https://cf-cc-qualitylab.com/product/buy-undetected-canadian-dollars/

It is incredibly average to see the best inconspicuous components presented in a basic and seeing way Thank you. Actually, I run a site similar to yours. If you have time, could you visit my site? Please leave your comments after reading what I wrote. If you do so, I will actively reflect your opinion. I think it will be a great help to run my site. Have a good day.

https://cf-cc-qualitylab.com/product/paypal-flip/

https://cf-cc-qualitylab.com/product/western-union-flip/

https://cf-cc-qualitylab.com/product/cash-app-flip/ https://cf-cc-qualitylab.com/product/western-union-flip-sale/

https://cf-cc-qualitylab.com/product/legit-paypal-flip/

https://cf-cc-qualitylab.com/product/legit-cash-app-flip/ https://cf-cc-qualitylab.com/product/buy-cloned-cards/

https://cf-cc-qualitylab.com/product/buy-credit-card-dumps-online/

https://cf-cc-qualitylab.com/product/cloned-credit-cards-for-sale/ https://cf-cc-qualitylab.com/product/legit-paypal-money-transfer/

https://cf-cc-qualitylab.com/product/legit-paypal-money-transfer/

https://cf-cc-qualitylab.com/product/western-union-money-flip/

https://cf-cc-qualitylab.com/product/hacked-cash-app/

Such an especially significant article. To a great degree charming to examine this article.I should need to thank you for the undertakings you had made for creating this astonishing article. https://cf-cc-qualitylab.com/product/automated-money-developer-machines/

https://cf-cc-qualitylab.com/product/high-quality-automatic-solution/ https://cf-cc-qualitylab.com/product/ssd-chemical-solution-packaging-materials/

https://cf-cc-qualitylab.com/product/vector-paste-ssd-solution/ https://cf-cc-qualitylab.com/product/humine-activation-powder/

https://cf-cc-qualitylab.com/product/99-99-pure-gallium-metal-alloy-for-mercury-replacement/ https://cf-cc-qualitylab.com/product/automated-money-developer-machines/

I am another client of this site so here I saw different articles and posts posted by this site,I inquisitive more enthusiasm for some of them trust you will give more data on this points in your next articles.

https://cf-cc-qualitylab.com/2024/02/01/western-union-uberweisen-umdrehen/ https://cf-cc-qualitylab.com/2024/02/01/kaufen-sie-western-union-umdrehen/

https://cf-cc-qualitylab.com/product/automated-money-developer-machines/

https://cf-cc-qualitylab.com/2024/02/01/koop-western-union-omzet/

https://cf-cc-qualitylab.com/2024/01/31/acquistare-lanciare-western-union/

https://cf-cc-qualitylab.com/2024/01/31/contanti-ribaltabili-della-western-union/

https://cf-cc-qualitylab.com/2024/01/30/volteo-de-dinero-de-western-union/

https://cf-cc-qualitylab.com/2024/01/30/comprar-voltear-de-western-union/

https://cf-cc-qualitylab.com/2024/01/28/transfert-dargent-western-union/

https://cf-cc-qualitylab.com/2024/01/28/retournement-de-western-union/

https://cf-cc-qualitylab.com/2024/01/27/online-kredietkaart-dumps-kopen/

https://cf-cc-qualitylab.com/2024/01/27/gekloonde-kredietkaarten-te-koop/

https://cf-cc-qualitylab.com/2024/01/27/gekloonde-kaarten-kopen/ https://cf-cc-qualitylab.com/2024/01/27/kaufen-sie-kreditkarten-dumps-online/

https://cf-cc-qualitylab.com/2024/01/27/geklonte-karten-dumps-zum-verkauf/

All things considered I read it yesterday yet I had a few musings about it and today I needed to peruse it again in light of the fact that it is extremely elegantly composed.

https://cf-cc-qualitylab.com/2024/01/27/kaufen-sie-geklonte-karten/

https://cf-cc-qualitylab.com/2024/01/27/acquistare-scarico-di-carte-di-credito-in-linea/ https://cf-cc-qualitylab.com/2024/01/27/carte-di-credito-clonate-in-vendita/ https://cf-cc-qualitylab.com/2024/01/27/carte-clonate-in-vendita/ https://cf-cc-qualitylab.com/2024/01/26/comprar-volcados-de-tarjetas-de-credito/

https://cf-cc-qualitylab.com/2024/01/26/tarjetas-de-credito-clonadas-a-la-venta/ https://cf-cc-qualitylab.com/2024/01/25/comprar-tarjetas-clonadas/ https://cf-cc-qualitylab.com/2024/01/25/acheter-des-dumps-de-cartes-de-credit-en-ligne/

https://cf-cc-qualitylab.com/2024/01/25/cartes-de-credit-clonees-a-vendre/

https://cf-cc-qualitylab.com/2024/01/25/cartes-clonees-a-vendre/ https://cf-cc-qualitylab.com/2024/01/23/requisitengeld-zum-verkauf/ https://cf-cc-qualitylab.com/2024/01/23/falschgeld-kaufen/

https://cf-cc-qualitylab.com/2024/01/23/gefalschter-euro-zum-verkauf/

https://cf-cc-qualitylab.com/2024/01/21/koop-valse-polymeerbiljetten/ https://cf-cc-qualitylab.com/2024/01/21/koop-valse-bankbiljetten/

Actually, I run a site similar to yours. If you have time, could you visit my site? Please leave your comments after reading what I wrote. If you do so, I will actively reflect your opinion. I think it will be a great help to run my site. Have a good day.

https://cf-cc-qualitylab.com/2024/01/21/koop-valse-euro-bankbiljetten/

https://cf-cc-qualitylab.com/2024/01/21/comprar-dinero-falso-en-linea/

https://cf-cc-qualitylab.com/2024/01/21/comprar-dinero-falso/

https://cf-cc-qualitylab.com/2024/01/21/billetes-de-euro-a-la-venta/

https://cf-cc-qualitylab.com/2024/01/17/soldi-realistici-in-vendita/

https://cf-cc-qualitylab.com/2024/01/17/acquistare-denaro-falso/ https://cf-cc-qualitylab.com/2024/01/17/banconote-in-euro-in-vendita/ https://cf-cc-qualitylab.com/2024/01/17/argent-accessoire-a-vendre/ https://cf-cc-qualitylab.com/2024/01/17/acheter-des-billets-indetectables/ https://cf-cc-qualitylab.com/2024/01/17/faux-euros-a-vendre-en-ligne/

https://cf-cc-qualitylab.com/2024/01/09/buy-euros-online-cyprus/

https://cf-cc-qualitylab.com/2024/01/09/fake-polymer-notes-for-sale/

https://cf-cc-qualitylab.com/2024/01/09/counterfeit-money-for-sale/

https://cf-cc-qualitylab.com/2024/01/09/buy-fake-dollar-bills/

https://cf-cc-qualitylab.com/2024/01/09/buy-prop-money-canada/

https://cf-cc-qualitylab.com/2024/01/09/buy-fake-polymer-notes/

Pinkuy, 2024/04/11 07:07

I'm constantly amazed at the variety of wordle words I can come up with to fit the grid.

Solar, 2024/06/06 08:24

I appreciate the clarity of your writing. Thanks!

태국 유흥, 2024/06/16 01:20

Are you ready to dive into the vibrant world of Thailand nightlife? Find out more: →→ 태국 유흥

Things to do, 2024/07/19 12:43

Have you ever had a boring weekend and want to have fun but out of ideas? Have you ever been to a new city and want to discover it but don't know where to start? Let me introduce to you, Things to do post

베트남 에코걸 가격, 2024/08/19 05:55

Eager to experience Vietnam's dynamic nightlife? Read more: →→ <a href=“https://travellifeinvietnam.com/”>베트남 에코걸 가격</a>.COM

베트남 에코걸 가격, 2024/08/19 05:56

Eager to experience Vietnam's dynamic nightlife? Read more: →→ 베트남 에코걸 가격.Com

URL, 2024/08/26 10:37

Your blog posts are always top-notch. Keep it up! URL

URL, 2024/08/26 10:38

This article provided great insights. Thanks for sharing! https://1depositcasinonz.com/

replica watches, 2024/08/30 13:52

지투지, 2024/10/02 07:58

G2GBET에서 제공하는 최고의 배팅 플랫폼, 지투지를 지금 경험해 보세요.

Lucky cola, 2024/12/03 06:28

Challenge accepted? Enter the world of epic online games today!

seo, 2024/12/11 05:30

This is my first time i visit here. I found so many interesting stuff in your blog especially its discussion. From the tons of comments on your articles, I guess I am not the only one having all the enjoyment here keep up the good workYes i am totally agreed with this article and i just want say that this article is very nice and very informative article.I will make sure to be reading your blog more. Vibergirls

seo, 2024/12/11 05:36

Yes i am totally agreed with this article and i just want say that this article is very nice and very informative article.I will make sure to be reading your blog more. You made a good point but I can't help but wonder, what about the other side? !!!!!!ThanksI wanted to thank you for this excellent read!! I definitely loved every little bit of it. I have you bookmarked your site to check out the new stuff you post. Magazire

seo, 2024/12/11 05:38

Positive site, where did u come up with the information on this posting? I'm pleased I discovered it though, ill be checking back soon to find out what additional posts you include.Thank you because you have been willing to share information with us. we will always appreciate all you have done here because I know you are very concerned with our. NewsMagzi

seo, 2024/12/11 05:40

I’ve a undertaking that I am simply now operating on, and I have been at the look out for such info.This is my first visit to your web journal! We are a group of volunteers and new activities in the same specialty. Website gave us helpful data to work. News-Matrix

seo, 2024/12/11 05:41

Wow! Such an amazing and helpful post this is. I really really love it. It's so good and so awesome. I am just amazed. I hope that you continue to do your work like this in the future alsoYou make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers. News Matic

seo, 2024/12/11 05:43

i never know the use of adobe shadow until i saw this post. thank you for this! this is very helpful.I havent any word to appreciate this post…..Really i am impressed from this post….the person who create this post it was a great human..thanks for shared this with us.TechyIntel

seo, 2024/12/11 05:43

This type of message always inspiring and I prefer to read quality content, so happy to find good place to many here in the post, the writing is just great, thanks for the post.Yes i am totally agreed with this article and i just want say that this article is very nice and very informative article.I will make sure to be reading your blog more. You made a good point but I can't help but wonder, what about the other side? !!!!!!Thanks VentsBulletin

seo, 2024/12/11 05:44

Please continue this great work and I look forward to more of your awesome blog posts.You know your projects stand out of the herd. There is something special about them.This was really an interesting topic and I kinda agree with what you have mentioned here!I love seeing blog that understand the value of providing a quality resource for free.Hello, I have browsed most of your posts. This post is probably where I got the most useful information for my research. Thanks for posting, maybe we can see more on this. Are you aware of any other websites on this subject. Newzify

seo, 2024/12/11 05:48

Great write-up, I am a big believer in commenting on blogs to inform the blog writers know that they’ve added something worthwhile to the world wide web!..Hello, this weekend is good for me, since this time i am reading this enormous informative article here at my home. Vital-Mag

seo, 2024/12/11 05:50

I’ve a undertaking that I am simply now operating on, and I have been at the look out for such info.This is my first visit to your web journal! We are a group of volunteers and new activities in the same specialty. Website gave us helpful data to work. Silicon Insider

seo, 2024/12/11 05:50

Wow! Such an amazing and helpful post this is. I really really love it. It's so good and so awesome. I am just amazed. I hope that you continue to do your work like this in the future alsoYou make so many great points here that I read your article a couple of times. Your views are in accordance with my own for the most part. This is great content for your readers. Usefullideas

seo, 2024/12/11 05:52

I never know the use of adobe shadow until i saw this post. thank you for this! this is very helpful.I havent any word to appreciate this post…..Really i am impressed from this post….the person who create this post it was a great human..thanks for shared this with us. Technikerforscher

Real name:

E-Mail:

Website:

Enter your comment. Wiki syntax is allowed:

Please fill all the letters into the box to prove you're human. M W A B᠎ R Please keep this field empty:

Subscribe to comments

	S	M	T	W	F	T	S
51	15 15 15	16 16 16	17 17 17	18 18 18	19 19 19	20 20 20	21 21 21
52	22 22	23 23	24 24	25 25	26 26	27 27	28 28
01	29 29	30 30	31 31	01 01	02 02	03 03	04 04
02	05 05	06 06	07 07	08 08	09 09	10 10	11 11
03	12 12	13 13	14 14	15 15	16 16	17 17	18 18

Navigation Menu

Flight-Control

Hot Projects

SEEDStack

UCSSPM

picoReflow

PiGI

DIY ARA-2000

DSpace

Mission-Tags

Share & Donate

Elasticsearch Garbage Collector

Discussion

User Tools

Site Tools

Navigation Menu

Flight-Control

Hot Projects

SEEDStack

UCSSPM

picoReflow

PiGI

DIY ARA-2000

DSpace

Mission-Tags

Share & Donate

Elasticsearch Garbage Collector

Discussion

Page Tools