User Tools

Site Tools

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
mission:log:2013:07:17:elasticsearch-garbage-collector [2013/07/17 11:51] – created chronomission:log:2013:07:17:elasticsearch-garbage-collector [2014/02/20 15:05] (current) – Fixed command string to only cover logstash indexes chrono
Line 32: Line 32:
  
   if [ ${DF} -le 30000000 ]; then   if [ ${DF} -le 30000000 ]; then
-    INDEX=$(/bin/ls -1t /logdata/elasticsearch/nodes/0/indices/ | tail -1)+    INDEX=$(/bin/ls -1td /logdata/dntx-es/nodes/0/indices/logstash-* | tail -1 | xargs -n 1 basename)
     curl -XDELETE "http://localhost:9200/${INDEX}"     curl -XDELETE "http://localhost:9200/${INDEX}"
   fi   fi
Line 56: Line 56:
 </WRAP> </WRAP>
  
-It runs every 10 minutes (as a cron job) and checks the available space on the device where Elasticsearch stores its indices. In this example /logdata is mounted on /dev/md0. If md0 has less than 30gb of free diskspace it automagically finds the oldest Elasticsearch index and drops it via Elasticsearch's REST API. +It runs every 10 minutes (as a cron job) and checks the available space on the device where Elasticsearch stores its indices. In this example /dev/md0 is mounted on /logdata. If md0 has less than 30GB of free diskspace it automagically finds the oldest Elasticsearch index and drops it via Elasticsearch's REST API without service interruption (no stop/restart of Elasticsearch required). 
  
-A simple locking mechanism will prevent multiple running instances in case of timing issues. All you need is curl for it to work and it will increase your storage efficiency so that you can always have as much past data available as your storage allows without the risk of full disk issues or manual maintaining. +A simple locking mechanism will prevent multiple running instances in case of timing issues. All you need is curl for it to work and it will increase your storage efficiency so that you can always have as much past data available as your storage allows without the risk of full disk issues or the hassle of manual monitoring & maintaining. 
  
 {{tag>devops big-data monitoring elasticsearch software}} {{tag>devops big-data monitoring elasticsearch software}}
Trace:

Page Tools