Before an AI feature reaches production, list every data path it creates. Include user input, retrieved context, generated output, logs, telemetry, backups, and support access. The list is often more revealing than a product label.
Describe the minimum useful input
Ask what the feature actually needs to complete its task. Remove identifiers, narrow the context window, or give users a way to preview what will be sent before a request leaves the device.
Set retention and access rules
Document how long inputs and outputs remain available, which teams can inspect them, and how a deletion request travels through caches and backups. A short, specific rule is easier to test than a broad promise.
Make the boundary visible
Explain the data flow in product language. Users should be able to tell whether a feature is processing a document locally, sending it to a service, or using it to improve a future system.
Trust is easier to maintain when the data path can be described in one clear paragraph.
Start with a data-flow map, not a policy label
“Private,” “secure,” and “enterprise-ready” do not describe a data boundary on their own. A useful review follows a request from the moment a person selects text or uploads a file. It asks what enters the application, what is added by retrieval, where the request is processed, which service providers receive it, and what records remain after a response appears. The same map should include error telemetry, abuse monitoring, backups, and support tooling. These paths are easy to miss because they often sit outside the visible product experience.
The map should distinguish content from metadata. An assistant may not retain document text for training, yet a system can still retain timestamps, account identifiers, feature flags, prompt length, or diagnostic traces. Metadata can be essential for operating a service, but it still needs a stated purpose, retention period, and access rule. Listing it separately helps a team avoid an all-or-nothing discussion in which every operational record is treated as either harmless or forbidden.
Turn a boundary into design choices
Once inputs and outputs are named, teams can reduce unnecessary movement. A form can ask for a shorter excerpt rather than a full document. A retrieval layer can exclude fields that do not affect the answer. A client can show the selected content before submission. A workflow with sensitive material can route a request to a local or approved service rather than a general endpoint. None of these choices guarantees a particular regulatory result, but each is a concrete decision that can be tested and explained.
Retention needs the same specificity. “We keep data only as long as necessary” is meaningful only when a reader can find the event that starts the clock and the job that enforces deletion. Separate rules may be needed for request content, generated output, safety logs, account records, and backups. The review should also include access: whether support staff can inspect content, how privileged access is approved, and whether an exported audit record can show who accessed a request.
Test the exception paths
Most product diagrams describe the successful request. The difficult cases are a failed response, a user withdrawing a document, an account being closed, a model provider outage, or a support ticket that contains copied content. These situations reveal whether the published boundary matches the real operating path. A small test can trace one sample request through application logs and downstream systems, then confirm that the team's deletion and access procedures work as written.
For example, an internal writing assistant may retrieve approved policy pages and return a draft to an employee. The review should state whether the employee's prompt is stored, whether retrieved pages are logged, whether the draft enters a shared workspace, and whether an administrator can inspect any of those elements. The right answer will vary with the use case. The point is that the answer is available before a person relies on a vague privacy statement.
Questions for a release review
- What exact content and metadata leave the device for this feature?
- Which supplier, service, or internal team can process each category?
- What is the minimum input needed for a useful result?
- How long does each record remain, including diagnostics and backups?
- How can a user understand, correct, or delete the information where applicable?
Sources and further reading
Use the NIST Privacy Framework and the OWASP LLM application guidance when building a review checklist. The NIST AI Risk Management Framework provides complementary guidance for documenting context and governance.