In a sophisticated cyber intrusion targeting Linux systems, attackers exploited vulnerabilities in NVIDIA GPU telemetry, employing a range of advanced techniques to mask their cryptomining operations. This operation, tracked under the designation REF3364, executed a well-coordinated attack across three Linux GPU hosts after breaching an internet-exposed Ray job agent.
At the heart of this intrusion is Ray, a distributed computing framework that serves as a hub for tasks related to artificial intelligence training, inference, and Python workload management. Ray features a Jobs API that facilitates the submission of applications for execution on remote clusters, an aspect that may prove detrimental if the interfaces for job submission are left exposed to external access.
The compromised Ray dashboard, which typically listens for local commands on loopback, was found to have its job agent accessible from external networks. This weakness allowed attackers to submit jobs without any authentication barriers. Initial probing activities were detected on October 22, 2026. By October 24, the attackers had successfully executed a job that utilized the NVIDIA System Management Interface (nvidia-smi), effectively extracting the GPU inventory. The following day, they established a Python reverse shell that enabled interactive access to the compromised system.
This reverse shell initially operated with the privileges of the Ray session owner, allowing the attackers to escalate their access significantly. They proceeded to insert a new SSH authorized key, enabling them to gain root access. Following this, a robust toolkit was staged within the directory /opt/vadi_. However, investigators noted that the exact method by which privilege escalation was achieved remains unidentified.
A key feature of this operation includes an innovative evasion mechanism. Attackers replaced the legitimate NVIDIA Management Library (libnvidia-ml.so.1) with a modified wrapper. Instead of disabling monitoring entirely, the wrapper intercepts queries directed to the GPU, relays them to the authentic backend, and selectively alters the responses before providing them back to legitimate users. This clever tactic ensures that GPU utilization statistics appear normal, thus minimizing detection risk.
According to findings from Elastic Security Labs, the operation leverages SANSAR, a Rust-based orchestrator designed to oversee both CPU and GPU miners while granting operators an interactive shell interface for control. In essence, this orchestrator helps attackers manage their cryptomining setup while camouflaging their activities.
Upon inspection, it was determined that the GPU’s reported utilization remained at zero, while temperature readings were artificially randomized between 36 and 46 degrees Celsius, and power consumption metrics fluctuated between 100 and 200 watts. This manipulation extends to filtering process information, which impairs traditional monitoring tools like nvidia-smi that rely on NVIDIA’s Management Library (NVML).
Further enhancing their stealth, the attackers deployed a rootkit named acpi_thermal_mgr.ko—a hidden module installed through the Dynamic Kernel Module Support (DKMS) framework. This rootkit is capable of concealing configured processes, worker threads, files, and socket entries. It is designed to reload at system startup and utilizes hooks that suppress directory records, making inspection attempts yield errors that imply nonexistent processes or paths.
Communication between SANSAR and the attacker’s infrastructure utilizes a Rust implementation of Tor, known as Arti, which enables encrypted communication with an onion controller. Both direct connections and bridge-assisted connections feed into the same command service. The system supports various incoming message types, including registration and heartbeats, along with commands for interactive shell sessions, miner configurations, and TCP SYN traffic generation.
The attack utilized modified versions of popular mining software; CPU mining was conducted using XMRig, rebranded as “kurt,” while GPU mining employed SRBMiner-MULTI, referred to as “tilki.” Since the pool addresses and wallet credentials are dynamically obtained from the controller rather than being hard-coded, this adds an additional layer of complexity for detection.
Notably, the attackers also established a covert backdoor via the systemd service known as netns. This backdoor facilitated the retrieval of an encoded controller IPv4 address using calls to Ethereum smart contracts, allowing for direct TCP communications.
This incident serves as a stark reminder that seemingly idle GPU systems can harbor malicious activity. The telemetry interface being compromised casts doubt on the integrity of monitoring data. Consequently, investigations should not solely rely on utilization metrics but should also consider correlating various suspicious indicators, including Ray submissions, SSH persistence, alterations to NVIDIA libraries, DKMS artifacts, and any unexpected services that may have been enabled.
In summary, understanding the multi-layered tactics employed in this intrusion and the potential threats to system integrity is paramount for effective cybersecurity. Enhanced vigilance and independent monitoring mechanisms are essential to counter such sophisticated attacks that manipulate both kernel and user-space reporting simultaneously.