This January more than 1,100 NFL players and their agents found their personal data compromised and held for ransom. This became possible due to Elasticsearch security breach.
As it turned out in the report from the security technology provider Kromtech Security, nearly 4,000 Elasticsearch machines deployed on AWS have been infected with JackPOS and AlinaPOS malware. These clusters formed a huge botnet that was stealing and encrypting personal data like credit card details.
The malware disguised itself as a Java utility, saved itself in the %APPDATA% directory or any other subdirectory of Java-related software. The malware was infecting the other machines, sending the encrypted data to the culprits in the process, or demanding ransom for releasing the data back to the rightful owner.
How was this even possible? As it came out, the ES installation packages below version 6.0 created 3 default users: elastic, logstash and kibana, with “changeme” passwords. These passwords were obviously intended to be replaced with strong personal passwords at once, yet many admins slacked and forgot about this procedure.
As a result, a whopping 4,000 of Elasticsearch machines (with 99% of them being deployed on AWS) was infected. AWS has a detailed ES setup/configuration guide, yet it seems most of the admins simply clicked Next, Next, Next, OK or chose the quick installation option, omitting the security configuration as a whole. The situation was elaborated in more details in the Elastic article.
How to secure your Elastic cluster?
Elastic did not stand idle and published a blog post with instructions on securing the Elasticsearch machines. The very first recommendation was to update to the latest ES version, obviously… The company went to great lengths to remove the possibility of admins skipping the initial password setup.
For example, TLS connection is now required even inside the cluster and checking that it is set up properly is now the part of the bootstrapping process. In addition, a security setting called “xpack.security.authc.accept_default_password” prohibits using the default password on a production ES cluster. Finally, a configured CLI tool automatically generates random bootstrap passwords for all the ES default users to eliminate the chance of the hacker gaining access in the short time between installing the cluster and issuing a strong permanent password.
We recommend making the following steps in order to protect your ES cluster:
- Update your ES to the latest version (6.0 at the time of writing this article). Reinstalling ES from scratch would be even better
- Block the external access to all the ports you do not use
- Whitelist the trusted IPs
- Check and monitor all logs, connections and network traffic on all of your servers
- After any suspicion of the security breach, it’s better to reinstall all the compromised machines at once to avoid minute monitoring of each separate one for months.
As mundane and banal as they seem, these actions will greatly increase the chance to secure your sensitive data and avoid Elasticsearch malware infection.
Final thoughts on the ways to secure your Elastic cluster
At IT Svit we were aware of the vulnerable default passwords from the start and went an extra mile to secure our clients. In all the projects that demand using the Elastic cluster, we leverage a third-party authentication module to prevent any potential security breach through weak ES authentication protocols.
Are you sure your admins did not skip the security configuration while deploying the Elasticsearch cluster to AWS? Are your passwords strong enough? Ensure the security of your customer’s sensitive data and your safety of mind by following the simple steps above!
Initially, I’ve posted this story on my company’s blog — https://itsvit.com/blog/will-elastic-cluster-hacked/