Best Paper Award at CAIS 2026

Florian Holzbauer, David Schmidt, Gabriel K. Gegenhuber, Sebastian Schrittwieser, and Johanna Ullrich received the Best Paper Award at the AgentSkills Workshop at the ACM Conference on AI and Agentic Systems (CAIS 2026) for their work “Context Matters: Repository-Aware Security Analysis of the Agent Skill Ecosystem.”

The paper, studies the emerging ecosystem of agent skills: extensions that add functionality to local AI agents such as Claude Code and OpenClaw. As these skills become increasingly popular and are distributed through dedicated marketplaces as well as GitHub, their security properties are becoming an important concern for developers, users, and platform operators.

In their study, we collected and analyzed 238,180 unique skills from three major distribution platforms and GitHub. Their repository-aware analysis shows that scanner reports can substantially overestimate maliciousness when skills are assessed in isolation. By considering whether a flagged skill is consistent with its surrounding GitHub project, the number of suspicious skills drops to only 0.52%. At the same time, the paper identifies previously undocumented real-world attack vectors, including the hijacking of skills hosted in abandoned GitHub repositories.

Abstract

Agent skills extend local AI agents, such as Claude Code and OpenClaw, with additional functionality. Their growing popularity has led to dedicated marketplaces resembling mobile app stores, as well as automated scanners that assess whether skills are benign or malicious. However, scanner reports from individual marketplaces classify up to 46.8% of skills as malicious, raising concerns about false positives. We present the largest empirical security analysis of the AI agent skill ecosystem to date.

We collect 238,180 unique skills from three major distribution platforms and GitHub, and analyze their contents, behavior, and repository context. Unlike existing scanner-based assessments, which evaluate skills largely in isolation, our repository-aware analysis checks whether a flagged skill is consistent with its surrounding GitHub project. This context substantially reduces the number of suspicious skills: only 0.52% remain suspicious after repository-aware analysis.

Our results show that existing scanners can substantially overestimate maliciousness when repository context is ignored. At the same time, we identify previously undocumented real-world attack vectors, including the hijacking of skills hosted in abandoned GitHub repositories. Overall, our findings provide a more robust view of the agent-skill ecosystem’s current risk surface and highlight the need for context-aware security evaluation.