TL;DR: An AI agent risk assessment framework scores what an agent can do through its tools, not what a model might say. It quantifies OAuth scopes, read versus write permissions, data classification, and blast radius across connected systems. Unlike generic model risk frameworks that focus on bias and explainability, agent-specific scoring identifies operational exposure from tool integrations. The framework applies to agents with write access to production systems or access to regulated data. Read-only, single-user agents handling public data fall outside the critical threshold. Composio enforces these controls at the infrastructure layer through policy-as-code validation, credential isolation with AES-256 encryption, and centralized audit logging that records denied calls alongside successful ones: enforced in the request path rather than through prompt instructions a model can reason around.
Most AI risk assessments start with the wrong question. They ask what the model might say. The question that matters is what the agent can do. A model that cannot act on external systems is a chatbot. An agent with write access to Salesforce, Slack, and GitHub is an operational risk that must be scored on permissions, data sensitivity, and blast radius.
This article provides a scoring template scoped to agents and shows how infrastructure-layer enforcement turns the assessment from a document into a control. The framework quantifies five dimensions: integration boundaries, read versus write access, data sensitivity classification, operational risk, and role mapping.
Why agent-specific risk assessments address a different attack surface
Generic AI risk frameworks evaluate model behavior, training data quality, and output explainability. NIST's AI Risk Management Framework structures assessment around four functions (Govern, Map, Measure, Manage) and seven characteristics of trustworthy AI: valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed.
For agents, the additional attack surface comes from OAuth tokens, API keys, and connected applications that grant access to external systems. The OWASP Top 10 for LLM Applications lists excessive agency as the sixth-highest risk for LLM-based applications in its 2025 edition, and it is the risk most specific to agent deployments.
Managing OAuth and tool permissions
OAuth scopes define the blast radius of delegated access. When attackers compromise a third-party application, the permissions granted to that application determine what they can access, from read-only data exfiltration to complete administrative control. Once an application receives a scope grant, the token carries the right to read, write, or administer resources until an admin revokes it or it expires.
A single over-scoped OAuth token can give an agent write access to every channel in your Slack workspace. An agent requesting broad workspace access during initial integration frequently receives more scope than the use case requires.
Composio's policy-as-code enforcement evaluates access restrictions in the request path before any model interaction. An admin disables the delete action for a Slack integration through the dashboard. The agent cannot delete regardless of what the prompt contains or what the model reasons.
Quantifying runtime tool permissions
Static documentation describes what permissions an agent should have. Runtime enforcement validates what permissions the agent actually exercises on each tool call. The distinction matters because OAuth grants issued during initial deployment frequently cover more than the integration needs, and permission scope drift accumulates as developers add capabilities without right-sizing access.
Composio encrypts connected-account credentials, auth configs, and API keys at rest using AES-256-GCM, and encrypts all traffic in transit using TLS, as documented in Composio's credential architecture. The agent runtime sends a request to the Composio proxy, the policy evaluator checks the action against the configured permission set, and on permit the credential resolves inside an isolated runtime to execute the outbound call.
This architecture validates permissions on every tool call, not just during quarterly access reviews. The centralized audit log records every tool call with user, team, tool, action, and outcome, including denied calls.
Quantifying data exposure beyond the model
Model risk frameworks evaluate training data quality and inference-time privacy controls. Agent risk assessment evaluates what data the agent can access through connected systems and where that data flows after retrieval.
For example, an agent with read access to Gmail and write access to Slack creates a data flow path from email content to team channels. If that agent processes a message containing PII or PHI and posts a summary to a public channel, the exposure is not a model failure, it is a permission scope failure that should have been scored during the risk assessment.
Agents that read from one system and write to another create data flow paths that must be mapped and governed. The risk assessment must score where data flows after the agent retrieves it. Read access to a sensitive system is lower risk if the agent only displays data to the user who owns it. Risk increases if the agent writes that data to a system with broader access controls.
Composio's platform compliance documentation specifies data retention controls and model training policies. Credentials are isolated from the LLM context and application code by design.
AI agent risk assessment template: scoring dimensions
A usable AI agent risk assessment template scores five dimensions on a 1-5 scale, where 1 represents minimal risk and 5 represents critical exposure requiring immediate mitigation or executive approval.
Scoring dimension | 1 (Minimal) | 3 (Moderate) | 5 (Critical) |
|---|---|---|---|
Integration boundary | Single app, read-only | 2-3 apps, limited write | 5+ apps, admin access |
Read vs write permissions | Read-only scopes | Write to non-prod systems | Write to production, delete actions |
Data sensitivity | Public data | Internal business data | PII, PHI, financial records |
Operational risk | No downstream automation | Triggers notifications | Modifies customer-facing systems |
Role mapping | Single-user access | Team-level access | Org-wide or cross-department |
Each organization will adjust thresholds based on data classification policies, regulatory requirements, and risk tolerance. The critical requirement is that scoring happens before deployment, not after an incident.
The following mappings are indicative points of reference, not direct equivalences or a determination that the EU AI Act applies to every agent deployment. The ISO control references use ISO/IEC 27001:2022 numbering.
Agent risk dimension | NIST AI RMF function (indicative) | EU AI Act (related provision) | ISO 27001 control (related) |
|---|---|---|---|
Integration boundary | Govern (access control policies) | A.5.15 Access control | |
Read vs write permissions | Manage (access restrictions) | A.5.18 Access rights and A.8.2 Privileged access rights | |
Data sensitivity | Map (data classification) | Art. 10: Data and data governance and GDPR where applicable | A.5.12 Classification of information |
Operational risk | Measure (impact assessment) | Art. 9: Risk management and Art. 27: Fundamental rights impact assessment where applicable | A.8.16 Monitoring activities and A.5.24 Information security incident management planning and preparation |
Role mapping | Govern (role-based access) | A.5.15 Access control and A.5.18 Access rights |
Defining read versus write permissions
Read permissions allow an agent to retrieve data. Write permissions allow an agent to modify, create, or delete data. The scoring scale differentiates read-only access (score 1-2) from write access to non-production systems (score 3) and write access to production systems with delete or bulk-update capabilities (score 4-5). Composio's policy-as-code enforcement allows admins to set permitted actions per user or role through the dashboard, enforced in the request path rather than documented in a policy PDF.
Mapping data sensitivity to risk
Data classification maps to regulatory exposure and breach notification requirements. Public data scores 1, internal business data scores 2-3, confidential data scores 4, and regulated data (PII, PHI, payment card data) scores 5. The scoring must account for data aggregation. An agent with read access to calendar metadata might not access message content, but calendar patterns can reveal sensitive business relationships. Score the most sensitive data class the agent can access, not the average.
Data residency requirements add complexity. For example, an agent processing EU customer data through a US-based SaaS application creates cross-border data transfer exposure. Composio has not publicly documented HIPAA BAA availability. Organizations requiring a HIPAA BAA should treat this as a disqualifying gap until that documentation is available.
Quantifying downstream operational risk
Operational risk scores what happens when the agent takes an action. An agent that reads data and generates a report for human review scores 1-2. An agent that sends emails scores 3. An agent that modifies production databases, triggers financial transactions, or updates customer-facing systems scores 4-5. The Drift breach illustrates blast radius multiplication: attackers used compromised OAuth credentials from the Salesloft Drift integration to access customer Salesforce environments and exfiltrate data.
Mapping agent access to user roles
Role-based access control applies to agents, not just human users. An agent acting on behalf of a single user with that user's permissions scores lower than an agent with service-account access across multiple users or departments. For example, an agent used by the sales team to update Salesforce scores 3. An agent used across sales, marketing, and customer success with access to all three departments' data scores 4-5. Composio Enterprise's SSO and SCIM integration maps directory groups to teams, allowing admins to configure permissions per role.
Calculating blast radius metrics
The five-dimension assessment uses a 1-5 rating scale, while the blast-radius calculation provides a separate numerical estimate of potential exposure.
Blast radius quantifies the maximum impact if an agent is compromised, misconfigured, or manipulated through adversarial prompts. The scoring model in this framework multiplies three factors:
Connected systems: Number of applications the agent can access
Permission depth: Read-only (1×), write to non-production (2×), write to production (3×), admin or delete (4×)
Data sensitivity: Public (1×), internal (2×), confidential (3×), regulated (4×)
For example, an agent with read access to Gmail and calendar (2 systems × 1× read × 2× internal data) has a blast radius of 4. An agent with write access to Salesforce, Slack, and GitHub (3 systems × 3× production write × 4× customer PII) has a blast radius of 36.
This exceeds an example threshold of 20 and would require CISO approval under this framework's example approval model.
Use the blast-radius score to determine which tool actions should be permitted, then enforce those access decisions through the policy controls described above.
Slack integration security audit steps
A Slack integration security audit validates that OAuth scopes match the documented use case and that policy controls enforce the intended permission boundary. This section is for the practitioner executing the audit on behalf of the IT owner.
Inventory granted scopes: List all OAuth scopes the Slack app has been granted. Common scopes include channels:history (read messages), chat:write (post messages), channels:read (list channels), and users:read (view user profiles).
Map scopes to use case: Document why each scope is necessary. For example, an agent that posts daily standup summaries needs chat:write for specific channels, not channels:history for all channels.
Test policy enforcement: Attempt an action outside the permitted scope. If the agent has chat:write but not chat:delete, verify that a delete request returns a policy violation error.
Review audit logs: Check the centralized audit log and observability layer for denied calls and anomalous patterns. For example, an agent attempting to access channels outside its configured scope may indicate misconfiguration, adversarial prompting, or multi-step workflow errors.
Recertify quarterly: Review granted scopes quarterly. Remove scopes that are no longer necessary as the use case evolves.
Auditing tool access for secure agent rollouts
Tool access auditing validates that agents have the minimum permissions necessary and that those permissions are enforced at runtime. The process follows four steps:
Inventory agent application dependencies: List every application each agent can access, including production integrations and development or staging connections. Document which user or service account owns each connection.
Audit OAuth scopes and API permissions: Pull the actual OAuth scopes granted during initial integration and compare them to the scopes documented in the risk assessment. For each scope, document the business justification.
Evaluate agent write access risks: Score not just whether the agent can write, but what happens if the write is incorrect or malicious. Implement human approval gates for high-risk write operations per OWASP recommendations for least agency.
Harden credential security and access: Isolate credentials from application code and the LLM context. Composio stores API keys as hashes and displays them once at creation. No recoverable copy exists on the platform. When a team member departs, credential enumeration through a centralized vault is straightforward because all credentials are stored in one governed location rather than scattered across developer environments and CI/CD secrets.
Enforcing least privilege for AI agents
Least privilege means agents have the minimum permissions necessary for their documented function. Enforcement happens at runtime through policy-as-code validation, which evaluates every tool call against configured permissions before executing the request. This differs from prompt-based guardrails that rely on the model's instruction-following capability. A prompt telling an agent not to delete records is a sign on a door. Policy-as-code is the lock.
Composio's policy enforcement evaluates access restrictions in the request path before any model interaction, so an admin's decision to disable delete actions holds regardless of what the prompt contains. Role-based access control applies the principle of least privilege to agent deployments. Sales team agents have access to Salesforce and Gmail. Engineering team agents have access to GitHub and Jira. SCIM integration maps directory groups to Composio teams, so permissions update automatically when users change roles.
Audit logs must record not just successful actions but denied attempts. Composio's audit logging records every tool call with user, team, tool, action, and outcome, including denied calls, providing the chain of custody evidence necessary for compliance audits and incident response.
Automating recurring security audits for agents
Manual security audits do not scale to organizations running many agents across many integrations, quarterly permission reviews become unmanageable as the catalog grows.
Assessment method | Time to audit | Evidence quality | Update frequency |
|---|---|---|---|
Static documentation review | 2-4 weeks per audit cycle | Point-in-time snapshot, manual verification required | Quarterly or annual |
Continuous monitoring with centralized audit logs | Real-time visibility, query results in minutes | Live data with denied-call records included | Continuous, with automated alerts |
Automated security audits reduce manual review hours while improving coverage across three operational areas:
Access recertification: Automated recertification prompts integration owners to confirm that OAuth scopes remain necessary as use cases evolve. Composio's centralized credential storage makes recertification straightforward because all credentials are visible in one dashboard.
Permission scope auditing: Compare granted scopes against actual usage. For example, an agent granted write access to Salesforce that has only performed read operations for 90 days is a candidate for scope reduction. The audit log provides usage data showing which actions each agent actually performs.
Rollback workflows: Automated rollback disables all OAuth tokens for a specific agent or integration when an incident occurs. Composio's centralized credential vault enables single-point revocation, terminating access to all connected applications simultaneously.
Book a call to walk through your security requirements before the next enterprise review. Contact our team to schedule a security architecture review.
FAQs
What is an AI agent risk assessment framework?
An AI agent risk assessment framework scores what an agent can do through its connected tools and applications. It quantifies OAuth scopes, read versus write permissions, data classification, integration boundaries, and blast radius. Unlike model risk frameworks that evaluate bias and explainability, agent risk frameworks identify operational exposure from tool permissions and credential access.
How does agent risk assessment differ from model risk assessment?
Model risk assessment evaluates training data quality, model bias, output explainability, and decision transparency. Agent risk assessment evaluates what the model can access and modify through OAuth tokens, API keys, and connected applications. An agent with write access to production systems creates operational risk that model behavior frameworks do not address.
What permissions should be included in an agent risk assessment template?
An agent risk assessment template must score OAuth scopes granted per integration, read versus write access per system, data sensitivity classification (public, internal, confidential, regulated), integration boundary scope (number of connected applications), and role-based access mapping (single-user, team, or organization-wide). Each dimension maps to specific enforcement controls.
How do you calculate blast radius for an AI agent?
In the scoring approach used here, blast radius equals connected systems multiplied by permission depth (read-only = 1×, write to non-production = 2×, write to production = 3×, admin or delete = 4×) and data sensitivity (public = 1×, internal = 2×, confidential = 3×, regulated = 4×). An agent with write access to three production systems containing customer PII has a blast radius of 3 × 3 × 4 = 36, which would require CISO approval under this framework's example approval model.
Who should approve high-risk agent deployments?
In an example approval model, high-risk agent deployments (blast radius above 20, write access to production systems with regulated data, or cross-department access) require CISO or VP of IT approval. Medium-risk deployments (blast radius 10-20) require director-level approval. Low-risk deployments (read-only access, single-user scope) can be approved by team leads with documented security review. Organizations should set approval thresholds according to their own risk policies.
Key terms glossary
Blast radius: Blast radius quantifies the maximum impact if an agent is compromised or malfunctions. In the scoring model used here, it is calculated by multiplying the number of connected systems by permission depth and data sensitivity.
OAuth scope: OAuth scopes define what actions a third-party application can perform on behalf of a user. Scopes determine the blast radius of delegated access because the token carries the right to read, write, or administer resources until revoked.
Least privilege: Least privilege means granting the minimum permissions necessary for a documented function. For AI agents, this means read-only access unless write access is specifically required, and scoped permissions rather than broad administrative access.
Policy-as-code: Policy-as-code enforcement evaluates access restrictions in the request path before the model is involved. Admins configure which actions are permitted per user or role, and those restrictions are enforced as code, not through prompt instructions the model can reason around.
Data classification: Data classification maps information to sensitivity levels (public, internal, confidential, regulated) that determine handling requirements and regulatory exposure. Agent risk assessments score the most sensitive data class the agent can access through connected systems.
Get started
