Compositional Policy Violations: When Step-Level Compliance Fails In Agentic AI Workflows

Problem

Compositional Policy Violations (CPVs) represent a significant gap in the evaluation of AI workflows, particularly when individual steps may pass compliance checks while the overall execution fails to adhere to governing policies. This issue is critical in agentic AI systems where ensuring compliance at every level is essential for safe and reliable operation. The authors note that existing frameworks do not adequately address these violations, leading to potential risks in deployment. The work is presented as a preprint and has not undergone peer review.

Method

The authors propose a comprehensive taxonomy of CPVs, identifying four distinct types: Authority Creep, Threshold Laundering, Cumulative Sum Violation, and Context Collapse. Each type represents a different mechanism through which compliance can be undermined despite individual steps appearing compliant. To address these issues, the authors introduce a provenance-aware runtime architecture designed to track and evaluate policies over complete execution traces rather than isolated steps. This architecture leverages raw provenance data to recompute guarded quantities, allowing for a more holistic assessment of compliance throughout the execution of AI workflows. The evaluation mechanism is specifically tailored to capture the nuances of CPVs, providing a framework for understanding how these violations manifest in practice.

Results

The available text does not report quantitative results.

Limitations

The authors acknowledge that step-level compliance checks are inherently limited in their ability to evaluate properties that are not determined by individual steps. This limitation suggests that while the proposed taxonomy and architecture can enhance understanding and detection of CPVs, they may not fully resolve the underlying issues of compliance in complex workflows. Additionally, the lack of empirical results in the paper raises questions about the practical applicability and effectiveness of the proposed methods in real-world scenarios.

Why it matters

This work has significant implications for the design and evaluation of AI systems, particularly those that operate in critical domains where compliance with policies is paramount. By highlighting the limitations of current compliance checks and introducing a structured approach to understanding CPVs, the authors pave the way for future research aimed at developing more robust compliance mechanisms. This could lead to safer and more reliable AI systems, particularly in applications where policy adherence is crucial.