To me, coding has always been trial and error. And by trial and error, I mean a considerable amount of errors. Today, I would like to share with you one of the mistakes I made when seting up Elasticsearch and how I solved it. This one is particularly remarcable because I got an email from the german Federal Office for Information Security (sounds scary, huh?) which totally freaked me out in the beginning and turned out to be just a friendly remainder at the end of the day.
Short story long
So, I´ve been working on seting up my server to run an R code that would basically open up a twitter stream and have the tweets stored into a database. I choose Elasticsearch for that purpose because it came in handy in combination with Logstash, which parses the tweets in order to push them into the database. And Kibana, the super good looking user interface for my Elasticsearch database, didn´t hurt either ;) Many use Elasticsearch, Logstash and Kibana as a trio, have a look at their website for further information.
Elasticsearch per default comes with no security settings at all, as the whole config file consists of a bunch of comments. If you just plan to play around with Elasticsearch, it´s perfectly fine to keep the default settings, but if you´re serious about it, you definitely need to secure it properly.
Because if you don´t and if you happen to have a server based in Germany, sooner rather than later you´ll get an abuse notification from the german Federal Office for Information Security. Take that. While in the US the first though when confronted with new apps, webapps and other services seems to be “Awesome! It´ll be so much fun to try it out!”, in Germany things are different. The first though on a German´s mind is rather “Holy crap, how about data privacy? Where are their servers hosted? Do they moneterize my data?”. Anyways, it took their bot a couple of hours to figure out that my Elasticsearch had a potential security flaw. I believe the Federal Office is checking for open 9200 ports with nmap or similar, but I can´t tell for sure. What I know is that I´ve spent the next day panicking about my server´s security and trying to find out if someone had hacked it by studying an insane amount of logs. It was so much fun! Not.
Buttom line here is that the german Federal Office for Information Security takes its job seriously and seems to have a service that automatically detects potential security gaps. Thank you guys for the friendly remainder! I keep on thinking it should not be named “abuse notification”, though. It make people feel like they´re cybercriminals or something went terribly wrong.
Where you should start
Once you have Elasticsearch installed, open the elasticsearch.yml file in an editor (under Linux it should be located under /etc/elasticsearch/) and scroll down to the Network And HTTP-section. You´ll see that per default, Elasticsearch binds itself to the 0.0.0.0 ip address. This can be a serious security concern, as it can make your database accessible to anyone. You can uncomment the network.host variable and change the value to either localhost or 127.0.0.1. I prefer the latter, because this way I must be entirely sure that localhost acutally maps to 127.0.0.1. So, the setting in question would look like this: network.host: 127.0.0.1.
If you´re using Elasticsearch together with Logstash, don´t forget to change your Logstash config file, too! The output part should look something like that:
output {
elasticsearch {
bind_host => "127.0.0.1"
index => "whatever"
document_type => "whatever"
}
}Note: bind_host is what does the magic here.
Also, if you´re using Elasticsearch as a service, don´t forget to restart it with:
sudo service elasticsearch restartto make sure it´s grabing the new setting.
I´m wondering: Is there another country providing such a security service to their citizens?
Elasticsearch & the German Federal Office for Information Security was originally published by Kirill Pomogajko at Opiate for the masses on August 29, 2015.