I am pleased to share that our paper has been accepted to the 33rd ACM Conference on Computer and Communications Security (CCS 2026)!
As Large Language Model (LLM) agents increasingly automate offensive operations across the cyber kill chain, their efficacy in complex local post-exploitation tasks remains inadequately quantified. Among these, Linux privilege escalation is a key step between initial access and full system compromise. However, existing evaluations for this task are heavily limited by small sample sizes (typically fewer than 15 scenarios), lacking the scale required to distinguish genuine reasoning from simple pattern memorization.
In this work, we present PrivEscalate [1], the most comprehensive benchmark to date for measuring LLM-automated Linux privilege escalation. Our benchmark comprises 531 Dockerized scenarios spanning 14 distinct sub-categories. To rigorously probe generalization, we additionally derive 154 parameterized variant scenarios that inject distractors and environmental noise to test whether LLMs genuinely reason about exploitation or merely rely on surface-level pattern matching.
Evaluating four closed-source and two open-weight LLMs, our findings demonstrate that:
- Model capabilities are highly heterogeneous across vulnerability categories, with no single model dominating all areas, necessitating multi-dimensional risk assessments.
- LLMs are fragile under environmental perturbation, with only one of six models successfully generalizing across variants, making configuration rotation a highly effective defense against most models.
- Agent architectures can significantly amplify risk. We developed PrivEscAgent, a domain-specialized wrapper that augments a generic ReAct agent with deterministic enumeration, category matching, and step planning.
- PrivEscAgent improves upon prior baselines (such as hackingBuddyGPT) by 34 percentage points and elevates the weakest evaluated model above that baseline without requiring any underlying LLM modifications.
We are releasing PrivEscalate as an open-source, Dockerized measurement instrument to support both LLM agent evaluation and broader Linux privilege escalation research.
Congratulations to Yixuan, Zilong, and Yin on this exciting achievement! Looking forward to presenting our work at CCS 2026.
References
- Liu, Y., Zhen, Z., Wu, Y., & Li, Y. (2026, November). PrivEscalate: Measuring and Augmenting the Threat of LLM-Automated Linux Privilege Escalation. Proceedings of the 33rd ACM Conference on Computer and Communications Security (CCS).