- De-identification removes or replaces information that identifies people, so data can be licensed with lower privacy risk.
- It reduces risk but does not make data anonymous on its own, which is why licenses also ban re-identification.
- HIPAA recognizes two methods: safe harbor, which removes 18 identifier types, and expert determination.
- Replacing identifiers with consistent placeholders keeps text readable and useful for training.
De-identified, pseudonymized and anonymous
- De-identified: direct identifiers removed or replaced so the data cannot reasonably be linked to a person, under a stated standard.
- Pseudonymized: identifiers replaced with consistent codes; whoever holds the key can reverse it.
- Anonymous: cannot be linked to a person by anyone, by any reasonable means. Very hard to prove for rich data such as free text.
Methods by data type
Text (email, chat, tickets, documents)
Automated detection finds names, emails, phone numbers, addresses, account numbers and IDs. Replace each with a consistent placeholder ("Customer A", "Company B") so conversations still make sense. Then have people review a random sample to measure what was missed.
Audio
Mute or tone-replace spoken identifiers and payment details, de-identify transcripts the same way as text, and consider whether the voice itself identifies a person. Some buyers accept voice; others need voice conversion.
Images and video
Blur faces, license plates, screens and documents in frame. Remove EXIF location data. For medical images, scrub DICOM headers and burned-in text.
Structured records
Remove direct identifiers, generalize dates and locations, and check combinations (such as ZIP code, birth date and gender together) that could single someone out.
The HIPAA standard
For health information, HHS recognizes safe harbor (remove 18 types of identifiers with no actual knowledge that what remains could identify someone) and expert determination (a qualified expert certifies the risk is very small) (HHS).
The CCPA standard
California requires reasonable technical measures, a public commitment not to re-identify, and contractual commitments from recipients (Frankfurt Kurnit Klein & Selz).
Contract protections that go with it
- No re-identification or attempts to link with other data
- No onward transfer without consent
- Security requirements
- Deletion on request or at term end
- Notice if re-identification risk is discovered
Who should do it
Sellers should select or jointly select the de-identification provider, approve the protocol and require certification, rather than handing raw data to the buyer to clean (Frankfurt Kurnit Klein & Selz).
Questions
Can the buyer de-identify the data for us?
It's better if you or an independent provider do it before transfer. Handing raw data to a buyer to clean defeats the purpose.
Does de-identification reduce training value?
Done well, very little. Consistent placeholders preserve the structure that matters. Removing whole fields or messages costs more.