Choosing an AI vendor means deciding who will control part of your organization's working knowledge. Before you commit, you need to know where information travels, who can access it, how models change, and whether your team can keep working if the relationship ends. Those answers deserve the same attention as accuracy, speed, and price.
AI sovereignty means retaining meaningful authority over how AI operates within your organization. It includes knowing where information goes, deciding which systems may process it, investigating important actions, and moving to another arrangement when necessary. This guide translates that idea into a practical vendor interview for procurement, IT, security, and business owners. It applies to cloud, local, and hybrid deployments, including evaluations of KarmaBox.
This AI vendor sovereignty checklist turns those concerns into twelve questions across six areas, with evidence requirements and a practical scoring method. Use it to compare proposals, identify gaps, and make control a concrete part of your purchasing decision.
01Why a checklist
A polished demo answers whether software can complete a task. It rarely establishes who can change the model, inspect a private file, retain a conversation, or interrupt a workflow. Those questions become urgent after adoption, when colleagues have built processes around the product. A checklist brings them forward to the point where architecture, contract terms, and pricing can still change.
Start by describing one intended workload. Record the data categories, users, connected tools, deployment region, product tier, and consequences of failure. An assistant summarizing public reports needs different safeguards from an agent working with confidential contracts. Score the actual proposed configuration. A vendor's enterprise feature list says little about controls excluded from the plan you intend to purchase or disabled in your environment.
The regulatory background explains why evidence matters. The EU AI Act's Article 26 addresses deployer obligations for high-risk systems, including human oversight and monitoring. Its adopted text also addresses retention of automatically generated logs under deployer control, with conditions and exceptions. These are scoped duties, not requirements imposed identically on every AI tool [1].
DORA provides another procurement reference for financial entities within its scope. Article 30 addresses contractual provisions for ICT services, including locations, data recovery, incident assistance, and termination. Services supporting critical or important functions face additional provisions concerning audit and transition arrangements. Our questions borrow the practical discipline of asking for enforceable controls; the score itself is not a DORA assessment [2].
Use this guide alongside KarmaBox's introduction to sovereign AI for individuals and teams. That introduction explains the broader idea; this checklist asks what a particular supplier can substantiate. Give the questions to vendors before the meeting and invite technical and contractual owners. A salesperson should not have to improvise answers about subprocessors, retention exceptions, or migration commitments.
02Data residency
Data residency describes where data is stored or processed. Data sovereignty concerns the authority, rules, and practical controls governing that data. A regional hosting option can support sovereignty, but does not settle remote access, backup locations, model routing, or legal jurisdiction. Evaluate the complete information path, including services that sit outside the main application and features that users can activate later.
1. Where does every category of our data go?
Ask the vendor to map prompts, attachments, outputs, embeddings, telemetry, support tickets, and backups. Identify the storage location, inference location, relevant legal entity, and people able to access each category. Include failover: a regional service may change behavior during an outage. Request written confirmation of whether location restrictions apply to the whole workflow or only to selected storage resources.
A good answer identifies exceptions and demonstrates the configured routing. A dodge says only, "We host in Europe." For a pilot, use synthetic documents with recognizable markers and inspect available request records and network destinations with your technical team. Treat this as a limited test, not proof of every backend process. Reconcile the observed behavior with the architecture and contractual commitments.
2. What are your permitted uses and retention periods?
Separate delivery of the service from training, evaluation, abuse monitoring, product improvement, and human review. Ask which settings apply by default and whether permissions cover all organizational users. Require a retention schedule for each data category, including backups and support records. Clarify what deletion removes immediately, what expires later, and which legal or security exceptions allow continued retention.
A good answer provides a purpose-and-retention schedule tied to the agreement. A dodge substitutes "You own your data" for processing restrictions. Examine definitions carefully: OpenAI's published DPA defines its covered Customer Data as personal data processed on the customer's behalf. Confidential business material outside that definition needs review under the relevant agreement too [3].
03Model portability
Model portability is the ability to move a workload between suitable models without losing essential behavior, controls, or business records. It does not require every model to produce identical answers. It requires a documented migration path, measurable acceptance criteria, and a realistic understanding of dependencies. A menu containing multiple models provides choice, but does not by itself demonstrate that your workflow can move safely.
3. Can we change models without rebuilding the workflow?
Request an inventory of provider-specific dependencies: prompt formats, tool calls, structured outputs, embeddings, retrieval indexes, fine-tuning, and safety settings. Ask what exports, what needs conversion, and what requires rebuilding. Model weights may be unavailable under a hosted service, and an open-weight license may impose conditions. Procurement should distinguish portable customer assets from proprietary vendor technology and third-party licensed components.
A good answer demonstrates the same sample workflow on an alternative model and records quality, latency, cost, and permission differences. A dodge points to an "OpenAI-compatible" endpoint as complete proof. Use a small, representative evaluation set with expected outcomes and unacceptable errors. Review KarmaBox documentation while preparing product-specific questions, then verify the selected deployment rather than assuming all documented options are included.
4. Who controls model changes and retirement?
Ask about version identification, material-change notices, evaluation windows, rollback, and end-of-life support. Establish what happens if a provider removes a model or changes its safety behavior. Where version pinning is unavailable, require another workable change-management process. Record who approves a migration and whether the vendor can silently switch providers during load spikes or incidents. Sensitive workflows should have a defined response to unexpected routing.
A good answer defines notice, testing responsibilities, and a fallback that your team can operate. A dodge promises "the newest model automatically." An upgraded model may still change classification, citation style, or tool behavior enough to disrupt a business process. Review changes against your own acceptance criteria. Better general benchmark performance does not establish suitability for your documents, languages, and decision boundaries.
04Audit access
Audit access means being able to obtain evidence about relevant controls and reconstruct important events. A certification or assurance report can help, but its scope, date, exclusions, and covered products matter. Buyers also need practical access to their own operational records. Define the questions an investigation must answer before deciding which logs and reports are sufficient for the proposed workload.
5. Can we reconstruct a consequential AI action?
Ask whether records identify the user or agent, time, model version, connected tool, permission decision, and human approval where applicable. Request export formats, retention controls, access restrictions, and protection against unauthorized alteration. Avoid collecting sensitive prompt content by default merely because it might be useful. The logging design should support investigation while respecting data minimization and the organization's retention policy.
A good answer lets your team reconstruct a sample action from exported evidence. A dodge offers only a dashboard screenshot. The EU AI Act's high-risk deployer provisions make human oversight and operational monitoring particularly relevant in covered settings, but logging is useful beyond those settings. It should reveal whether the actual workflow respected the permissions and review steps your organization approved.
6. What investigation and incident rights do we receive?
Request the relevant independent assurance materials and the contractual process for additional questions, inspections, and regulator cooperation where needed. Define incident categories, notification triggers, escalation contacts, and assistance costs. Distinguish personal-data breaches from outages, unauthorized tool actions, and model failures. One notification clause may not cover all four, and your internal response deadlines may be shorter than the vendor's reporting commitment.
A good answer links incident definitions to named contacts and a tested response process. A dodge says "We are certified." For comparison, Anthropic's published DPA includes a 48-hour outer notification limit after awareness of its defined Security Breach. Read that definition and the applicable contract instead of treating the number as a promise about every operational incident [4].
05Subprocessors
Subprocessors are organizations engaged to perform relevant personal-data processing on a processor's behalf. Your operational dependency map may need to go further, covering model hosts, monitoring tools, integration services, and support providers. Ask which participants receive content, metadata, or administrative access. A supplier list without processing purposes and locations makes it difficult to understand the exposure created by a particular workflow.
7. Which third parties can process or access our information?
Request a current list with legal names, purposes, locations, data categories, and the features that trigger involvement. Ask whether enabling web search, analytics, connectors, or support changes the list. Identify contractual flow-down obligations and responsibility for failures. Separate subprocessors from independent third-party services your users choose to connect, because responsibilities and applicable terms may differ between those arrangements.
A good answer traces your sample workflow through the relevant providers and identifies optional services. A dodge says "We use trusted partners." Trust is not an inventory. For a local deployment, ask the same questions about telemetry, update checks, crash reporting, and remote assistance. Keeping inference on a device does not establish that every supporting service remains local or receives no identifying information.
8. What can we do when a subprocessor changes?
Ask how notices arrive, when access begins, how long objections remain possible, and what remedies follow an unresolved objection. Check whether you must subscribe to updates yourself. A contractual right that nobody monitors is weak operational protection. Assign a mailbox and owner, and record whether the remedy is a different service route, a feature restriction, or termination with defined financial consequences.
A good answer explains both the notice mechanism and the practical remedy. A dodge says "The list is on our website." Public DPA samples illustrate why comparison matters: OpenAI describes a 30-day objection period and a resolution process; Anthropic describes a 15-day objection period for specified concerns. These are examples from published terms, not universal standards or substitutes for reviewing your executed agreement [3][4].
06Legal-demand handling
Legal-demand handling concerns how a vendor responds when an authority seeks customer information or access. Data location alone cannot answer the question. Relevant entities, applicable laws, operational access, and encryption architecture all matter. No supplier can credibly promise that a contract makes every binding legal demand disappear. The useful questions concern process, transparency, scope, and the technical ability to disclose readable information.
9. How do you validate and respond to disclosure demands?
Ask which entity receives demands, who reviews validity and scope, when the customer receives notice, and what happens when notice is prohibited. Request the policy on challenging improper or overbroad demands where lawful, minimizing disclosure, and preserving a record. Aggregate transparency reporting can provide context, but does not guarantee the outcome of a future case or establish that your particular data cannot be reached.
A good answer describes a bounded process and acknowledges legal restrictions on notification. A dodge promises "Government access is impossible." For sensitive workloads, have qualified counsel assess the relevant jurisdictions and transfer arrangements. The procurement team can gather the evidence and identify uncertainty; it should not turn a vendor's marketing statement into a legal conclusion about immunity from disclosure.
10. Who can decrypt or retrieve the information?
Map encryption keys, key administrators, application access, support privileges, and recovery processes. Customer-managed keys may improve control, but the label alone does not prove the provider cannot access plaintext during inference. Ask what the system must decrypt to operate and where that happens. Examine logs and backups separately, since they may use different protection and recovery arrangements from the primary content store.
A good answer explains normal access, emergency access, and the consequences of key revocation. A dodge says only "Encrypted at rest and in transit." Test revocation with synthetic material in a nonproduction environment and record what remains accessible. Consider the availability trade-off too: removing access or destroying keys may interrupt workflows or make recovery impossible, so ownership must include a responsible recovery plan.
07Exit and export
Exit capability is the ability to recover useful assets and continue an acceptable level of work after leaving a vendor. A cancel button proves very little about that capability. Plan for an orderly migration and an unexpected interruption. Identify assets that matter to continuity, how quickly they must be recoverable, and which dependencies a replacement system would need to reproduce.
11. Can we export everything needed to continue working?
Request sample exports covering source documents, conversation history, prompts, workflow definitions, permissions, evaluations, and logs where available. Identify excluded assets and explain why they cannot move. Embedding vectors may be incompatible with another model; rebuilding an index requires source material and configuration. Proprietary model weights are not automatically customer assets. A complete inventory is more useful than an unqualified promise of "full portability."
A good answer imports the export into another usable environment and verifies sample records and permissions. A dodge offers a PDF of conversations. Evaluate documentation, format stability, rate limits, export duration, and specialist labor. For a critical workflow, measure the time to restore a minimum usable service. An archive that opens successfully can still be insufficient to resume the actual business process.
12. What survives termination or supplier failure?
Ask about retrieval windows, migration assistance, fees, notice periods, continued service during transition, and deletion evidence. Include insolvency or discontinuation, not only voluntary cancellation. Establish which commitments survive termination and which depend on another provider. A promise of assistance has limited value if availability, scope, and price remain unspecified until the moment your organization needs to leave.
A good answer provides a costed transition plan and explains deletion exceptions. A dodge repeats "Cancel anytime." DORA's Article 30 includes data recovery and return provisions, with additional transition requirements for services supporting critical or important functions. Even outside its scope, buyers can use the underlying procurement question: can the organization keep operating through a supplier change without relying on last-minute goodwill [2]?
08How to score the answers
Score each question from zero to three, for a maximum of 36. Award zero for an absent or unacceptable answer; one for a specific explanation with material evidence gaps; two for adequate documentation and applicable contractual commitments; and three when those commitments are supported by relevant operational evidence. Record limitations even when awarding full points. A successful demonstration verifies its tested scope, not every possible future scenario.
One rung = one thing the vendor has produced. A point measures evidence, not impressions — and limitations get recorded even at full points.
Each of the six categories contains two questions and has a maximum of six points. Keep the category scores visible beside the total. Before interviews, mark mandatory controls and the evidence required to pass them. If a requirement fails, record a blocked approval or a formally authorized exception. Do not use strong answers about data location to offset an inability to retrieve essential business records.
Consider HarborPilot AI, a fictional internal knowledge assistant. In this illustrative assessment, data residency scores 5/6, model portability 3/6, audit access 5/6, subprocessors 4/6, legal-demand handling 4/6, and exit 1/6. The total is 22/36. These invented scores demonstrate the method; they are not measured results for KarmaBox or any other real vendor, and they do not represent an industry benchmark.
22 out of 36 looks survivable until you read the last row. Exit and export was mandatory, it failed, and no total can rescue the approval. HarborPilot AI is a fictional vendor.
| Category | Illustrative score | Note |
|---|---|---|
| Data residency | 5/6 | Routing documented; failover behavior still a gap |
| Model portability | 3/6 | Alternatives exist; migration path not demonstrated |
| Audit access | 5/6 | Evidence exports; retention settings need confirmation |
| Subprocessors | 4/6 | List provided; objection remedy unspecified |
| Legal-demand handling | 4/6 | Bounded process; jurisdiction assessment pending counsel |
| Exit and export | 1/6 | Mandatory requirement failed — approval blocked |
| Total | 22/36 | The total cannot offset a failed mandatory requirement |
HarborPilot AI is a fictional vendor. These scores demonstrate the method and are not measured results for any real supplier.
The buyer has designated successful export and recovery as mandatory. Approval therefore remains blocked despite reasonable scores elsewhere. The remediation plan requires a usable export, a documented recovery test, and priced transition assistance before production use. There is no universal passing score. Set thresholds around workload consequences, use the same evidence standard across bidders, and rescore after material architecture, contract, or subprocessor changes.
09Keep a scoring record
Use a scorecard to record all twelve answers, evidence references, scores, mandatory requirements, remediation owners, and review dates. Keep one copy for each vendor and proposed configuration. Its purpose is to preserve the reasoning behind the decision, so a colleague can understand the assessment months after the sales meeting.
Before completing the worksheet, gather the service agreement, DPA, architecture diagram, subprocessor list, incident procedure, export documentation, and relevant assurance reports. Ask the vendor to identify which document supports each answer. For KarmaBox evaluations, compare the proposed deployment with the KarmaBox service terms, then resolve any requirements that need a specific contractual commitment. Apply the same scrutiny to every supplier.
10Turn the interview into an approval decision
Run the review in three steps. First, collect written answers and identify missing evidence before scheduling demonstrations. Second, test the controls that matter most to the workload using synthetic or approved data. Third, reconcile the demonstration with the agreement. If the sales presentation, technical documentation, and contract conflict, record the discrepancy and resolve it before relying on the capability.
Assign responsibilities inside your organization as carefully as you assess the vendor. Procurement should maintain commercial commitments, security should evaluate access and investigation evidence, technical owners should test routing and migration, and legal advisers should assess applicable obligations. A business owner should decide whether residual limitations are acceptable for the intended process. The vendor cannot make that organizational risk decision on your behalf.
Use a concrete test scenario. For example, create a synthetic project with documents, role restrictions, an agent action requiring approval, and a small evaluation set. Follow it from upload through model execution, audit review, export, and deletion. This reveals gaps between individual product features. A vendor may support permissions and export separately while failing to preserve the same restrictions after migration.
Keep an evidence register with the document version, product configuration, test date, owner, and unresolved limitation. Link each score to that register. When a statement changes from "planned" to "available," verify availability in your environment before increasing the score. Likewise, when a contract renews, check that previously negotiated protections survive instead of copying last year's assessment into a new purchasing decision.
Finally, keep the approved scope visible to users. State which data may enter the system, which connectors are permitted, when human review is required, and who can authorize a change. A strong vendor agreement cannot prevent every employee from choosing an unsafe configuration. Sovereignty becomes practical when purchasing commitments, technical settings, and everyday working instructions describe the same operating boundary.
11Frequently asked questions
Is data residency the same as AI sovereignty?
No. Data residency concerns location. AI sovereignty also includes access authority, model choice, supplier dependencies, oversight, legal exposure, and exit capability. Regional hosting can support control, but buyers must verify the rest of the workflow and the applicable commitments.
Can a cloud AI vendor meet sovereignty requirements?
Yes, depending on the workload and required controls. Evaluate processing locations, access, permitted data use, change management, and exit arrangements. Local deployment also needs scrutiny because telemetry, updates, support, and optional cloud features can introduce external dependencies.
What is a passing sovereignty score?
There is no universal passing score. This framework totals 36 points, but mandatory requirements take priority over the total. Define approval thresholds before comparing vendors and require evidence appropriate to the consequences of the intended workload.
Does a DPA prove that an AI vendor is sovereign?
No. A DPA addresses defined processing obligations. Buyers must also examine the service agreement, architecture, actual configuration, access evidence, and exit process. Check the definitions, exceptions, and products covered before treating any provision as an organization-wide guarantee.
Does this checklist establish EU AI Act or DORA compliance?
No. It is a procurement tool, not legal advice or certification. Applicable duties depend on the entity, use case, system classification, jurisdiction, and operative rules. Use qualified legal and technical review to determine obligations and assess the supporting evidence.
12About the publisher
KarmaBox publishes this guide to help teams ask concrete questions about AI control and vendor dependence. Apply the checklist to KarmaBox as well as alternative providers. The scoring method is an editorial procurement framework; the fictional example is illustrative. Product suitability and legal compliance require assessment of the actual deployment and applicable agreements.
Source review date: 2 November 2026. The legal discussion references the cited adopted texts. Check operative amendments, commencement rules, and the terms that apply to your agreement before making a compliance decision.