Soft Deletion
The “soft delete” anti-pattern describes the approach where, instead of properly deleting something from a database, the database schema is changed to contain a “deleted” field that is then marked as “true” to signal the deletion. In the article, some remedies are offered, ranging from “just delete it already” over lifecycle-aware database and application design to history-aware data warehousing.
I’d like to reflect on cases when the anti-pattern might actually be a pattern, i.e., a rather advisable approach. In the GDPR context, companies are frequently facing deletion orders, wherein users require the companies to delete personal data. There’s no easy way around that, those data have to be deleted. However, records vanishing from the database can harm the company’s ability to run historically correct analytics. This is why anonymisation or pseudonymisation are often applied in such cases.
Anonymisation removes any trace of personal data and “nulls it out”, e.g., by replacing it with dummy entries such as “+00 000 0000000” for a phone number or “ABC DEF” for a name. That’s fine, and also quite OK from the GDPR perspective. It may go a step too far though, because what different users do will be lost - all users collapse into one if, e.g., an e-mail address or phone number are a primary key. The impact on analytics can be harsh.
Therefore, pseudonymisation is preferable. Ideally, a user record has a primary key that’s not one of the fields containing personal information. A UUID can be used. The personal fields can then be “nulled out” as far as it makes sense. For example, if the phone number is relevant because the user’s country plays a role in analytics, the phone number bits following the country code can be deleted.
Note that there’s a fine line to tread here. The GDPR is pretty clear on customer rights. Companies’ legitimate interests in retaining their analytical abilities come in second. Therefore, considerations like the aforementioned one for phone numbers can be made for data fields, but protecting the customers’ anonymity takes precedence. If a customer’s identity can be reverse engineered, that’s a no-go.
All in all, customer databases should be designed carefully, and prepared for deletion by pseudonymisation, from the get go. In the age of GDPR, there’s no excuse for doing otherwise.