Just got back from configuring some log storage for 10 TB so let’s talk sharding 😀 Node limitations Main source: The definitive guide to elasticsearch HEAP: 32 GB at most: If the heap is less than 32 GB, the JVM can use compressed pointers, which saves a lot of memory: 4 bytes per pointer instead of…
Continue readingElasticSearch
Linux basic commands
To get the list of file with permission: ls -l <foldername> To get the list of folder and files present in folder ls <foldername> To get resource status about cpu,memory, storage space free -m To give read write premission on folder chmod -R 777 <folderpath> (for read / write ) where -R is recursively apply…
Continue readingElasticsearch Administration
Important elastic queries: GET / GET _nodes/stats GET _cat/nodes?v GET /_cat/health?v GET /_cat/shards/matpa?v GET _cat/indices?v GET _cat/recovery?v GET /_cat/shards/matpa?v GET /_cat/thread_pool GET _cluster/settings GET /_cluster/allocation/explain GET _cluster/health?level=nodes GET _cluster/health?level=indices GET /_cluster/allocation/explain?include_disk_info=true GET _nodes?filter_path=**. GET /_nodes/thread_pool GET /_nodes?pretty&filter_path=nodes.*.settings.path.data” GET /_all/_alias/* Settings PUT _cluster/settings { “transient”: { “cluster.routing.allocation.include._ip”:”ip_address1″ } } PUT /my_temp_index { “settings”: { “number_of_shards”…
Continue readingElasticsearch rolling upgrade
Step 1: Disable shard allocation PUT _cluster/settings { “transient”: { “cluster.routing.allocation.enable”: “none” } } Step 2: Stop non-essential indexing and perform a synced flush (Optional) POST _flush/synced Step 3: Stop and upgrade a single node # stop elasticsearch and kibana sudo service elasticsearch stop sudo service kibana stop # remove plugins, if installed sudo bin/plugin…
Continue readingSteps to install elasticseach on linux
1. Installing Oracle Java – Go to root shell sudo -i – Run these commands in this order (https://www.digitalocean.com/community/tutorials/how-to-install-java-on-ubuntu-with-apt-get) 1. apt-get install python-software-properties 2. add-apt-repository ppa:webupd8team/java 3. apt-get update 4. apt-get install oracle-java8-installer 5. vim /etc/environment 6. Add this line – JAVA_HOME=”/usr/lib/jvm/java-8-oracle/” 7. source /etc/environment 8. echo $JAVA_HOME 2. Installing Elastic Search (https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html) 1. Download…
Continue readingTerminology used in Elastic
In Elasticsearch, a document belongs to a type, and those types live inside an index. You can draw some (rough) parallels to a traditional relational database: Relational DB ⇒ Databases ⇒ Tables ⇒ Rows ⇒ Columns Elasticsearch ⇒ Index⇒ Types ⇒ Documents ⇒ Fields Note: In Elasticseach version 6 or more we need to create…
Continue readingELASTICSEARCH ON WINDOWS – INSTALLATION
ELASTICSEARCH – INSTALLATION We are using Elasticsearch for our search functionality on the new API framework we are developing to replace our legacy applications and I’ve started reading “ElasticSearch Server” by Rafal Kuc and Marek Rogozinski and published by PACKT. I’ve played about with Elasticsearch and have installed it locally but only know the basics of posting…
Continue reading