Researchers have discovered a concerning issue with hundreds of thousands of servers running Prometheus open-source monitoring software on the open web. These servers are exposing passwords, tokens, and creating opportunities for denial of service (DoS) and remote code execution.
Prometheus is a widely used open-source observability tool that organizations use to monitor the performance of their applications and cloud infrastructure. However, there is a catch. According to the documentation, untrusted users have access to the Prometheus HTTP endpoint, which grants them access to all time series information contained in the database, as well as operational and debugging information.
Despite these warnings, many users seem unaware of the default exposure of Prometheus or underestimate the value of the data that can be accessed. Using Shodan, researchers from Aqua Nautilus identified over 40,000 exposed Prometheus servers and more than 296,000 exposed “exporters,” which collect data from monitored endpoints. They found sensitive data and identified opportunities for “repojacking” and DoS attacks in these exposed servers and exporters.
When analyzing the data collected by Prometheus, seemingly harmless metrics like application performance, cloud tool usage, CPU, memory, and disk usage can reveal vulnerabilities that attackers can exploit. This includes plaintext passwords, tokens, and API addresses that should be kept secure. For example, the researchers discovered an exposed instance of Prometheus belonging to Skoda Auto, which exposed the company’s subdomains, docker registries, and images.
Apart from exposing sensitive information, open web Prometheus servers also pose a risk of DoS attacks. For instance, the ‘/debug/pprof’ endpoint, enabled by default in most Prometheus components, could be overloaded to disrupt communications or crash Amazon Web Services Elastic Compute Cloud (AWS EC2) instances or Kubernetes pods. Aqua Nautilus researchers successfully demonstrated this vulnerability, highlighting the potential impact of such attacks, even on critical systems like Kubernetes clusters.
To mitigate these risks, users are advised to take their Prometheus servers offline or implement authentication mechanisms. Tools are available to help address DoS vulnerabilities. However, another issue identified was the vulnerability of several Prometheus exporters to repojacking attacks. These attacks occur when a developer changes or deletes their GitHub account without retiring the associated namespace, allowing attackers to register the old username and plant malicious content under the same repository title.
The repojacking vulnerability was discovered in multiple exporters associated with claimable usernames, enabling potential remote code execution by attackers. Aqua Nautilus promptly reported this issue to Prometheus, leading to a resolution. However, the researchers warn that repojacking opportunities may be more widespread than realized, emphasizing the importance of monitoring project links to prevent malicious activity.
In conclusion, the exposure of Prometheus servers and exporters highlights the critical need for organizations to secure their monitoring tools and remain vigilant against potential cyber threats. By addressing authentication issues, mitigating DoS risks, and monitoring for repojacking vulnerabilities, organizations can protect their systems and data from malicious actors seeking to exploit these security gaps.
_luminous_Alamy.jpg?disable=upscale&width=1200&height=630&fit=crop)