compliance

Personally Identifiable Information (PII): What It Is and How to Handle It in AI Products

Any data that can be used to identify, locate, or contact an individual, including names, email addresses, IP addresses, and health information.

Every AI product that touches real users almost certainly processes personally identifiable information. PII is not just names and email addresses - it extends to IP addresses, device identifiers, behavioural patterns, location data, and any data point that, when combined with other information, can identify a specific individual. The challenge for AI product teams is that PII can enter your system in ways that are not immediately obvious: users paste personal details into LLM prompts, fine-tuning datasets contain personal references, and embeddings can encode identifying characteristics. Understanding what PII is, where it flows in your system, and how GDPR requires you to handle it is foundational to building AI products that enterprise customers will trust and regulators will not challenge. Under UK GDPR, the ICO can issue fines of up to GBP 17.5 million or 4% of global annual turnover for serious breaches, which means founders who treat data handling as an afterthought face real commercial exposure. Enterprise procurement teams, particularly in financial services and the NHS supply chain, now routinely ask for a GDPR data flow map before signing contracts. SpeedMVPs builds PII-aware data architecture from sprint one, giving founders a documented data flow and a defensible compliance position ready for enterprise due diligence within the standard two to three week delivery window.

What Counts as PII Under GDPR

GDPR uses the term personal data rather than PII, and defines it broadly as any information relating to an identified or identifiable natural person. The practical scope is wider than most founders expect. Names, email addresses, phone numbers, and national insurance numbers are obvious examples. So are IP addresses, cookie identifiers, device fingerprints, and advertising IDs - these are routinely collected by analytics tools and are personal data under GDPR. Health information, biometric data, genetic data, data revealing religious beliefs, political opinions, racial or ethnic origin, sexual orientation, and trade union membership are special categories of personal data under Article 9, carrying stricter processing requirements. For AI products, the critical area is inference: if your model can infer a protected characteristic from other data points - inferring likely health conditions from purchase history, for example - you may be processing special category data even if you never explicitly collected it.

Where PII Enters AI Systems

The entry points for PII in AI products are more numerous than in traditional software. User-generated prompts often contain personal information, sometimes deliberately and sometimes incidentally. A user asking your AI to help draft a redundancy letter might paste in employee names, salaries, and performance records. A customer using your AI-powered support tool might include order numbers that link to personal accounts. If those prompts are logged for evaluation or fine-tuning purposes, you now have a PII retention problem. Training and fine-tuning datasets frequently contain personal information scraped from the web or pulled from internal systems. Embeddings generated from documents that contain personal data may preserve identifying information in vector form. Retrieval-Augmented Generation pipelines that index customer data or internal knowledge bases create a vector store that is, in legal terms, a personal data repository requiring GDPR-compliant data retention policies.

Lawful Basis for Processing PII

Under GDPR and UK GDPR, you cannot process personal data without a lawful basis. For AI products, the most common applicable bases are contract performance (you process data because it is necessary to deliver the service the user signed up for), legitimate interests (you process data for a purpose a reasonable person would expect and the interest is not overridden by the individual's rights), and consent (the individual has given clear, specific, and informed agreement). Consent is often the default assumption of founders, but it is also the most demanding to implement correctly: it must be freely given, specific, informed, unambiguous, and withdrawable. For analytics, LLM prompt logging, and model improvement purposes, you will typically need either explicit consent or a carefully documented legitimate interests assessment. Using special category data requires explicit consent or one of the narrower Article 9 exceptions.

PII Minimisation in Practice

The GDPR data minimisation principle requires collecting only what is necessary for the specified purpose. For AI products, this translates into a set of concrete design decisions. Do not log full prompt text if a hash or summary is sufficient for your evaluation purpose. Strip or pseudonymise PII from prompts before they are passed to third-party LLM APIs where possible. Implement prompt pre-processing that detects and redacts obvious PII patterns such as email formats, phone number patterns, and national insurance number formats before the data leaves your infrastructure. Define and enforce retention periods for all personal data, including prompt logs, conversation histories, and user-uploaded documents used in RAG systems. At SpeedMVPs, data minimisation architecture decisions are made during the discovery and scoping phase, not retrofitted after a compliance audit.

Data Subject Rights in AI Systems

GDPR gives individuals rights over their personal data that your AI product must technically support. The right of access means any user can request all personal data you hold about them. The right to erasure means they can ask for their data to be deleted. The right to rectification means incorrect data must be corrected. The right to restriction means processing can be paused while a dispute is resolved. The hard implementation challenge in AI is erasure. Deleting a user from your PostgreSQL database is straightforward. Removing their contributions from a fine-tuned model is not - it may require retraining. Removing documents they uploaded from a vector store is achievable if you have indexed them with user-level metadata. These are engineering problems with engineering solutions, but they must be considered during architecture design, not after a subject access request arrives.

Third-Party LLM APIs and PII Risk

When you send user data to a third-party LLM API such as OpenAI, Anthropic, or Google, you are engaging a data processor under GDPR. You must have a Data Processing Agreement in place with that provider. Most major LLM providers have DPAs available and do not use API-submitted data for training by default, but you should verify this in the current version of their documentation rather than assuming. You must disclose in your privacy notice that personal data may be processed by AI service providers. You must ensure the data transfer is lawful, which for EU personal data sent to a US provider requires Standard Contractual Clauses or equivalent safeguards. UK data transfers have separate adequacy rules under UK GDPR, which currently allow data flows to EEA countries as adequate. The ICO provides guidance on international data transfers that should be the starting point for UK-based teams.

Frequently Asked Questions

Is an IP address PII under GDPR?+

Yes. The Court of Justice of the EU confirmed in the Breyer case that dynamic IP addresses can constitute personal data when a provider has the legal means to obtain additional information to identify the user. For practical purposes, treat IP addresses as personal data in all GDPR-related planning. This applies to server logs, analytics data, rate limiting systems, and any other context where IP addresses are stored or processed.

Does sending user prompts to OpenAI violate GDPR?+

Not automatically, but it requires proper handling. You need a Data Processing Agreement with OpenAI, a lawful basis for the original processing of the user's data, a privacy notice that discloses AI processing, and for EU personal data, Standard Contractual Clauses to cover the transatlantic data transfer. OpenAI's API terms include a DPA and SCCs. You must ensure your own privacy documentation reflects this processing chain. If prompts contain special category data, additional safeguards apply.

How do I handle the right to erasure when I have used user data in a RAG system?+

For RAG systems, deletion of a user's documents from the vector store is technically achievable if you have indexed documents with user-level identifiers and can perform filtered deletions by user ID. This should be part of your data model from the start. For fine-tuned models that incorporated user data in training, erasure from the model weights is not currently technically feasible. In this case, you may need to demonstrate that the data cannot be extracted from the model and that the model's operation does not reveal the individual's data, and document this limitation in your DPIA.

What should my privacy notice say about AI processing?+

Your privacy notice should explicitly state that your product uses AI services, name the key third-party AI providers you use (such as OpenAI, Anthropic, or AWS Bedrock), describe what personal data may be included in AI processing, explain your lawful basis for that processing, and describe any international transfers of data. Users should understand that content they enter into AI features may be processed by external systems. Vague references to service providers are not sufficient under GDPR's transparency requirements.

Do I need a Data Protection Officer for my AI startup?+

Under GDPR, a DPO is mandatory for public authorities, organisations that carry out large-scale systematic monitoring of individuals, or organisations that process special categories of data at large scale. Most early-stage AI startups will not meet the threshold for mandatory appointment. However, the ICO recommends that organisations processing personal data at significant scale or in novel ways designate a DPO or equivalent competent person. If your AI product processes health, financial, or other sensitive data, taking specialist data protection advice is sensible even if formal DPO appointment is not legally required.

Building an AI product and want GDPR data handling right from day one? Get a free consultation at speedmvps.co.uk

Get a Free Quote