TL;DR: When your team evaluates an AI agent integration vendor, you need a question bank that goes beyond "are you SOC 2 compliant." The questions below are ready to paste and cover OAuth token storage, tenant isolation, least-privilege scope enforcement, audit log export, SOC 2 Type II evidence, ISO/IEC 27001 documentation, and NIST AI RMF alignment. Each section includes the specific evidence to request and the red flags that disqualify a vendor. Use it as a standalone checklist or adapt it to your existing third-party risk workflow.
Security teams wrote most vendor questionnaires before AI agents existed. Those questionnaires ask about firewalls and endpoint protection but skip the questions that matter when a third-party platform stores OAuth tokens on behalf of your users, executes actions across your SaaS stack, and decides which credentials to inject into outbound API calls.
Standard questionnaires miss the architecture that determines whether credentials are actually safe. This questionnaire covers the specific risks of agent integration platforms: credential vaulting, tenant isolation, scope minimization, audit export, and compliance evidence. Paste the questions into your review workflow as-is or adapt them to your organization's risk tiering.
Why standard vendor questionnaires miss AI agent integration risks
Standard vendor security questionnaires ask whether the vendor encrypts data at rest and whether they have a SOC 2 report. Those questions matter, but they don't probe the architecture that determines whether your credentials are actually safe.
An AI agent integration platform sits between your agents and every SaaS app those agents touch. It stores OAuth tokens for Gmail, Salesforce, Slack, GitHub, and whatever else your workflows require.
It executes actions on behalf of your users. If that platform has weak tenant isolation, a single query bug exposes every connected account across every customer. If it stores tokens alongside application code, a prompt injection attack can exfiltrate credentials through the LLM context.
OAuth token storage and credential handling
Token storage determines whether a breach exposes one password or live refresh tokens for every connected app across your organization.
Questions to include in your assessment:
Encryption at rest: Do you encrypt all stored OAuth tokens (access and refresh) with AES-256 or stronger? Is encryption applied at the field level, the database level, or both? The standard pattern is AES-256-GCM (specified in NIST SP 800-38D) with a managed KMS key stored separately from the ciphertext, consistent with the key management guidance in NIST SP 800-57.
Key management: Who has access to production encryption keys? Are keys stored in a dedicated KMS or HashiCorp Vault, separate from the database? NIST SP 800-57 Part 1 Rev. 5 (Recommendation for Key Management) calls for limiting access to cryptographic keys to the people and systems that need them, so database administrators, developers, and analysts without a need should have no access to production keys.
Token isolation from application code: Can your application servers or the LLM context access raw provider tokens? Look for an architecture that decrypts credentials inside an isolated runtime and injects them into outbound requests without exposing them to the calling application or LLM context.
Refresh token protection: Third-party provider tokens (Google, Salesforce, Slack) must be sent back to the provider, so the vendor has to store them with reversible encryption. Ask for envelope encryption with keys held in a KMS. For credentials the vendor issues itself, such as its own API keys or session tokens, ask whether it stores only a hash. When a provider rotates refresh tokens, does the platform store the new token right away and handle reuse detection? RFC 9700 §4.14 (OAuth 2.0 Security Best Current Practice) recommends sender-constrained or rotating refresh tokens to prevent replay. The provider’s authorization server enforces this, not the integration vendor.
TLS in transit: Do you enforce TLS 1.2 at minimum (TLS 1.3 preferred) for all API communication, including internal service-to-service calls? NIST SP 800-52 Rev. 2 requires TLS 1.2 as the floor and support for TLS 1.3.
Red flag: The vendor describes token storage as "encrypted" but can't name the cipher, the key management system, or the isolation boundary between credentials and application code.
Composio encrypts connected-account credentials, auth configs and API keys at rest with AES-256-GCM. When an agent calls a tool, Composio decrypts the credential inside an isolated runtime, injects it into the outbound HTTP request, and returns only the API response. The credential never reaches the model or your application layer.
Tenant isolation in multi-tenant platforms
Most AI agent integration platforms run multi-tenant architectures. Your credentials, configurations, and audit logs share infrastructure with every other customer. The question isn't whether the vendor is multi-tenant. It's whether the vendor enforces tenant isolation architecturally or assumes it holds.
Questions to include in your assessment:
Isolation model: Do you use logical isolation (shared database with tenant-scoped queries), physical isolation (separate databases per tenant), or a hybrid model? Row-level security (RLS) policies enforced with the FORCE option can provide strong logical isolation at lower infrastructure cost than separate databases, but only when they apply to every query path.
Tenant context enforcement: Is tenant scoping enforced at every data access path, including background jobs, webhook handlers, and internal admin tools? A single code path that skips tenant validation creates a cross-tenant data leak.
Cross-tenant testing: Have you conducted penetration tests specifically targeting cross-tenant data access? Can you share the methodology and remediation evidence?
Credential isolation across tenants: Are encryption keys tenant-specific, or does a single master key encrypt all tenants' credentials? Tenant-specific key envelopes limit the blast radius of a key compromise.
Red flag: The vendor says "we use row-level security" but can't show you a policy definition, an audit log proving enforcement, or a penetration test result validating isolation.
Least-privilege OAuth scope enforcement
An agent integration platform that requests broad OAuth scopes ("full access to Gmail") when narrow scopes suffice ("read-only access to a specific label") multiplies your risk surface. Every unnecessary scope is additional exposure if the vendor or the agent is compromised.
Questions to include in your assessment:
Scope minimization: Does your platform request only the minimum OAuth scopes required for each integration action? Can you provide a scope-to-action mapping for your top integrations?
Scope auditability: Can our team audit which OAuth scopes are active for each connected account? Can we revoke or narrow scopes at runtime without breaking core functionality?
Per-action enforcement: Does your platform validate that each agent action uses only the scopes explicitly granted for that action? NIST SP 800-53 Rev. 5, control AC-6 (Least Privilege) requires that users, and processes acting on their behalf, get only the access they need for their assigned tasks. That includes agents and the permissions necessary for their function.
Admin-level scope controls: Can our administrators set policies that restrict which actions an agent can perform, regardless of the OAuth scopes granted by the user? This is the difference between a prompt instruction ("don't delete anything") and a policy-as-code control that rejects delete actions in the request path.
Red flag: The vendor requests the broadest available scope for every integration "for simplicity" and has no mechanism for narrowing scopes per action or per user.
Composio enforces which toolkits are enabled, which accounts are connected, and what agents may do, with each rule evaluated in the request path before any model interaction. Scoped project API keys separate session management from tool execution, so these limits are enforced by the platform rather than by prompt instructions.
Audit log export and SIEM integration
Your security operations team needs to correlate agent activity with other security events. That requires structured, exportable audit logs with the fields your SIEM expects. A vendor that logs activity but can't export it in a usable format leaves you blind during incident investigation.
Minimum requirements to include in your AI vendor risk assessment questionnaire:
Requirement | What to ask | What good looks like |
|---|---|---|
Log fields | What fields does each audit log entry contain? | Each entry records the actor (user or service identity), the action, a UTC timestamp, the resource accessed, the result (success or failure), the connected account, and any denied calls |
Export methods | How can we export audit logs? | Webhook (real-time), S3 sync (batch), Syslog/HTTPS endpoint, or native SIEM connector (Splunk, Datadog) |
Retention | How long are audit logs retained? | At least 12 months total, with the most recent 3 months immediately searchable (the PCI DSS v4.0 Requirement 10.5.1 benchmark) |
Tamper resistance | Are logs cryptographically signed or append-only? | HMAC-SHA256 signing or S3 Object Lock to prevent retroactive modification |
Denied call logging | Do you log actions that were denied by policy? | Yes, with the policy that triggered the denial. Denied-call logging is what makes logs usable as compliance evidence rather than activity summaries. |
Query API | Can we query logs programmatically by time range, actor, resource, or action? | RESTful endpoint with filtering and pagination |
Red flag: The vendor provides a dashboard for viewing logs but has no export API, no webhook support, and no documentation of log schema.
Composio logs every tool call with user, team, tool, action, and outcome, including denied calls. This creates a complete chain of custody for every non-human identity interaction across your integrated apps.
SOC 2 Type II and ISO 27001 evidence: What to request and how to validate
A compliance badge on a vendor's website tells you almost nothing. You need the report, the observation period, the controls tested, and the auditor's findings.
Framework | What to request |
|---|---|
SOC 2 Type II | Full report under NDA, observation period, Trust Services Criteria covered, exceptions and deviations |
ISO 27001 | Current Statement of Applicability (SoA), ISMS scope statement, certification body accreditation, most recent surveillance audit |
Questions to include in your assessment:
For SOC 2 Type II:
Is your report Type I or Type II? If Type I, when will Type II be available?
Which Trust Services Criteria does the report cover?
Will you share the complete report under NDA?
Does the report contain any exceptions or control deviations? How were they remediated?
For ISO 27001:
Can you provide your current, dated, version-controlled SoA? ISO/IEC 27001:2022 Clause 6.1.3(d) requires certified organizations to produce an SoA that records which Annex A controls apply, whether each is implemented, and why any are excluded.
What systems, locations, and data categories does your ISO 27001 certification cover?
Is your certification body accredited by a recognized national authority?
Can you provide evidence of your most recent surveillance audit, including any nonconformities and corrective actions? Under Clause 6.1.3(d), the SoA is the primary record of which controls the vendor has actually implemented, so compare the audit findings against it.
Composio holds SOC 2 Type II and ISO/IEC 27001:2022 certifications, with pre-filled compliance documentation that reduces the evidence-gathering burden during your review. Composio's compliance and data handling documentation covers data retention, model training policies, and credential handling in detail.
NIST AI RMF and ISO/IEC 42001 alignment
AI agent integration vendors operate at the intersection of traditional information security and AI-specific risk. NIST AI RMF 1.0 provides a voluntary framework for identifying and managing AI-related risks across four functions: Govern, Map, Measure, and Manage. ISO/IEC 42001:2023 is the first international standard for AI management systems, and it follows the same Annex SL high-level structure as ISO/IEC 27001, so vendors with an existing ISMS can map their current controls and audit evidence to it.
Questions mapped to NIST AI RMF functions:
NIST AI RMF function | Question | Evidence to request |
|---|---|---|
Govern | Does your organization have documented AI governance policies, defined roles, and executive accountability for AI risk? | AI governance policy, roles and responsibilities, evidence of executive oversight |
Map | Have you documented the context in which your AI system operates, including intended use, data provenance, and third-party dependencies? | System inventory, threat model, data lineage documentation |
Measure | Do you monitor AI system behavior in production with metrics tracked over time? | Monitoring dashboards, drift detection logs, incident records |
Manage | Are identified AI risks prioritized and addressed through a formal risk treatment plan? | Risk register, incident response playbook for AI-specific incidents (prompt injection, unauthorized scope escalation) |
Questions mapped to ISO/IEC 42001 Annex A controls:
AI system impact assessment (A.5): Does the vendor perform impact assessments at planned intervals and when systems change materially? Clause 8.4 of ISO/IEC 42001:2023 requires organizations to run AI system impact assessments.
AI system life cycle (A.6): Does the vendor document responsible design, development, verification, and deployment at each stage?
Data quality for AI systems (A.7): Does the vendor define and document data quality requirements? Annex A of ISO/IEC 42001:2023 lists the full control catalog.
Third-party relationships (A.10): Does the vendor assess and monitor its own AI supply chain, including model providers and sub-processors?
Red flag: The vendor hasn't heard of NIST AI RMF or ISO/IEC 42001, or claims alignment without documented evidence of which controls are implemented.
Data retention, deletion, and model training policies
Your data protection obligations don't stop at your own systems. When an AI agent integration vendor processes personal data on your behalf, you need contractual guarantees on retention limits, deletion SLAs, and model training exclusions.
Questions to include in your assessment:
Retention limits: What is your maximum data retention period for agent activity logs, customer data, and stored credentials? GDPR Article 5(1)(e) requires that personal data be kept no longer than necessary for its intended purpose.
Deletion SLA: What is your data deletion SLA after we request deletion? Can you delete data from backups, or only from live systems? Under the CCPA (Cal. Civ. Code §1798.105) and its regulations (§7051), service providers must help businesses carry out consumer deletion requests and must be bound to this by contract.
Model training: Do you use our data or our customers' data for model training, fine-tuning, or evaluation in any form? Can we opt out with a contractual clause?
Sub-processor flow-down: How do you ensure sub-processors comply with our deletion requests? Do your sub-processor agreements include flow-down deletion obligations? GDPR Article 28(4) requires processors to pass the same data-protection obligations down to their sub-processors. DPAs should include deletion obligations, timelines, confirmation requirements, audit rights, and retention restrictions.
Data Processing Agreement: Do you have a signed DPA (GDPR Article 28) covering deletion obligations, timelines, and confirmation requirements?
Red flag: The vendor has no documented retention schedule, can't commit to a deletion SLA, or reserves the right to use customer data for model training without an explicit opt-out.
Breach notification and incident response
When a vendor that stores your OAuth tokens gets breached, the notification timeline determines how quickly you can rotate credentials, assess exposure, and meet your own regulatory obligations.
Questions to include in your assessment:
Notification timeline: What is your maximum time from breach discovery to customer notification? GDPR Article 33 gives controllers 72 hours to notify the supervisory authority, and Article 33(2) requires processors to notify the controller “without undue delay.” Your vendor agreement should specify a shorter window for customer notification (24 to 48 hours for critical incidents).
AI-specific incident response: Does your incident response plan address AI-specific incidents: prompt injection leading to unauthorized actions, model drift causing unintended data access, or compromise of stored agent credentials? Under NIST AI RMF 1.0, the Manage function requires organizations to prioritize and respond to AI risks identified through monitoring.
Forensic support: Will you provide technical incident details, forensic logs, and access records to help us meet our own regulatory notification obligations?
Credential rotation commitment: In the event of a credential store compromise, what is your process and timeline for rotating all affected tokens?
Red flag: The vendor's incident response plan is generic, doesn't address credential compromise scenarios, or has no committed notification timeline in the DPA or MSA.
Existing frameworks you can adapt
You don't have to build your question bank from scratch. Several recognized frameworks provide structured questionnaires you can adapt for AI agent vendor assessment.
Framework | Source | Scope | AI coverage | Best for |
|---|---|---|---|---|
CAIQ v4.1 | Cloud Security Alliance | 283 cloud security questions mapped to CSA Cloud Controls Matrix (CCM) v4.1, released January 27, 2026. The earlier v4.0.x version has 261 questions | Limited. The CSA released AI-CAIQ separately | Standardized security questionnaire and self-assessment for cloud vendors |
AI-CAIQ v1.1 (2026) | Cloud Security Alliance | 320 questions mapped to the 247 controls of the AI Controls Matrix (AICM) v1.1 | Comprehensive. Mapped to ISO/IEC 42001 (closely aligned), plus gap mappings to NIST AI RMF | AI 600-1 and the EU AI Act |
CAIQ-Lite | Cloud Security Alliance | 73 condensed questions (based on CCM v3.0.1) | No | Fast-track for lower-risk vendors |
SIG Lite / SIG Core | Shared Assessments | ~330 (Lite) to 800+ (Core) questions, varying by version | SIG AI module available | Regulated industries: financial services, healthcare |
NIST AI RMF Playbook | NIST | Governance-focused guidance with crosswalks to ISO 42001 and EU AI Act | Comprehensive | Organizations building internal AI governance |
For AI agent integration vendors specifically, start with AI-CAIQ as the base framework, then add the credential-specific questions from the OAuth token storage, tenant isolation, scope enforcement, and audit log sections of this article. AI-CAIQ covers AI governance broadly but doesn't probe token storage architecture or per-action scope enforcement at the depth your review requires.
How Composio answers these questions
When your team sends Composio a security questionnaire, the answers come back within a day in most cases because Composio documents them in advance rather than assembling them per-request.
Credential storage: Composio encrypts all credentials with AES-256-GCM in a centralized vault, isolated from application code and LLM context. Composio stores API keys as hashes and displays each key once, at creation, so no recoverable copy exists on the platform.
Tenant isolation: Composio isolates resources by organization and project: data in one project isn’t visible from another.
Scope enforcement: Composio's policy-as-code controls evaluate rules in the request path before any model interaction. Admins define permitted actions per user, team, or tool. Agents cannot widen their own access.
Tool-execution logging: Composio exposes per-call tool execution logs through a Logs API, which can be pulled on a schedule into Splunk, Datadog or another SIEM.
SOC 2 Type II and ISO/IEC 27001:2022: Composio holds both certifications.
Compliance documentation: Composio provides pre-filled compliance packs covering common frameworks. Composio's platform compliance documentation covers data retention, model training policies, and sub-processor details.
If you're evaluating Composio or any other agent integration vendor, book a call with Composio's security team and bring your questionnaire.
FAQs
What is an AI vendor security questionnaire?
An AI vendor security questionnaire is a structured set of questions that IT and security teams use to evaluate the security posture of a vendor providing AI-powered services, covering credential handling, tenant isolation, access controls, audit logging, compliance certifications, and AI-specific risks like prompt injection. It probes the architecture that determines whether your data and credentials are protected at the infrastructure layer, not just generic security claims.
What should an AI vendor risk assessment questionnaire include for agent integration platforms?
Cover OAuth token storage architecture (encryption, key management, isolation from application code), tenant isolation model, least-privilege scope enforcement, audit log export (fields, retention, denied-call logging), SOC 2 Type II and ISO 27001 evidence, NIST AI RMF and ISO/IEC 42001 alignment, data retention and deletion SLAs, model training opt-out, and breach notification timelines. Each area should include specific architectural questions, not generic security claims.
How do I verify a vendor's SOC 2 Type II report?
Request the full report under NDA (not an executive summary), confirm it's Type II (testing operating effectiveness over time) rather than Type I (design at a single point), verify the observation period covers at least 6 to 12 months, check which Trust Services Criteria are covered, and review any exceptions or control deviations noted by the auditor.
What is the difference between CAIQ and AI-CAIQ?
CAIQ (Consensus Assessments Initiative Questionnaire) is the Cloud Security Alliance’s standardized security questionnaire for cloud vendors. CAIQ v4.1 (released January 27, 2026) has 283 questions mapped to the Cloud Controls Matrix v4.1: the earlier v4.0.x version has 261. AI-CAIQ (v1.1, July 2026) is CSA’s AI-specific questionnaire. Its 320 questions map to the AI Controls Matrix, which has gap mappings to ISO 42001, NIST AI RMF / AI 600-1 and the EU AI Act. Use AI-CAIQ for AI agent integration vendors.
How do I assess tenant isolation in a multi-tenant AI platform?
Ask the vendor to describe their isolation model (logical, physical, or hybrid), whether tenant scoping is enforced at every data access path including background jobs and admin tools, whether they've conducted cross-tenant penetration tests, and whether encryption keys are tenant-specific or shared across all customers.
What ISO 27001 evidence should I request from a vendor?
Request the current Statement of Applicability (listing all 93 Annex A controls with inclusion/exclusion decisions), the ISMS scope statement, the certification body's accreditation status, and evidence of the most recent surveillance audit including any nonconformities and corrective actions.
What breach notification timeline should I require from an AI vendor?
Require a maximum of 24 to 48 hours from breach discovery to customer notification for critical incidents, with a contractual commitment to provide forensic logs, technical details, and a credential rotation plan within a defined SLA. GDPR Article 33 sets 72 hours for supervisory authority notification, so your vendor notification window must be shorter to give you time to comply.
Key terms glossary
Credential isolation: An architectural pattern where OAuth tokens are stored and decrypted in a separate, access-controlled environment, never exposed to application code or the LLM context window.
Least privilege: The principle that users and agents should hold only the minimum permissions necessary to perform their function, as defined in NIST SP 800-53 Rev. 5, control AC-6.
Non-human identity: A machine identity (service account, OAuth token, API key) that accesses systems on behalf of users or automated processes, requiring the same governance as human identities.
Policy-as-code: Access restrictions defined as executable code and enforced in the request path before any model interaction, rather than as prompt instructions a model can reason around.
Statement of Applicability (SoA): A mandatory ISO/IEC 27001:2022 document (Clause 6.1.3) listing all 93 Annex A controls with implementation status and justification for each inclusion or exclusion.
Tenant isolation: The architectural enforcement preventing one customer's data, credentials, or configurations from being accessible to another customer in a multi-tenant system.
Get started
