Privacy and data protection

Why privacy matters to data people

Data scientists handle phone numbers, addresses, purchase histories, health records and location trails. Misuse or a leak can cause fraud, harassment, discrimination and loss of trust, and can bring heavy penalties for the organisation. Good privacy practice is not a legal afterthought; it shapes which data you collect, how you store it and what you build.

Key ideas

• Personal data: information about an individual who can be identified from it, directly (name, phone, Aadhaar number) or indirectly (a combination of pin code, birth date and gender).
• Consent and purpose limitation: use data only for the specific purpose people were told about and agreed to.
• Data minimisation: collect and keep only what you need. If age group is enough, do not store date of birth.
• Storage limitation: delete data when the purpose is over.
• Security: encryption, access controls, and logs of who accessed what.

Anonymisation versus pseudonymisation

Pseudonymisation replaces direct identifiers with codes, such as replacing customer IDs with salted hashes. Whoever holds the key or the mapping can link the data back, so it is still personal data, but it reduces risk in daily analysis. Anonymisation means nobody can reasonably re-identify the person. It is much harder than it sounds: famous studies have shown that a combination like pin code, birth date and gender can single out most people in a population. Releasing "anonymised" data without testing this is risky.

import hashlib

def pseudonymise(customer_id, salt):
    return hashlib.sha256((salt + customer_id).encode()).hexdigest()[:16]

print(pseudonymise("CUST001", "s3cret"))

k-anonymity and differential privacy

k-anonymity is a simple check: every combination of quasi-identifiers (columns that could identify someone together, like age band, pin code and gender) must be shared by at least k people. If one row is the only "female, 70–80, pin 403001", she can be identified. Differential privacy is a stronger, mathematical approach: it adds carefully measured random noise to results so that no single person's data noticeably changes them. It is used by some census bureaus and large tech companies for published statistics.

The law in India: the DPDP Act, 2023

The Digital Personal Data Protection Act, 2023 governs the processing of digital personal data in India. Its key roles: the data principal (the person the data is about) and the data fiduciary (the organisation that decides why and how data is processed). Fiduciaries generally need valid consent given through a clear notice, must use data only for the stated purpose, must protect it, must report breaches, and must let people access, correct and erase their data. A Data Protection Board handles complaints and can impose large penalties. The government notified the detailed DPDP Rules in 2025 with a phased timeline; check the current rules and official guidance before relying on any detail.

If you handle data of people in the European Union, the GDPR may apply too. Sector rules also matter: for example, RBI rules for payment data and specific rules for health data. Aadhaar numbers have their own legal protections; never store or display full Aadhaar numbers without a clear legal basis, and mask them (show only the last four digits) where display is needed.

Common mistakes

• Copying full production customer data to a laptop "for analysis". Use masked or minimal extracts.
• Calling data anonymous after removing names only.
• Pasting personal data into AI tools that are not approved for it.
• Keeping data forever "in case it is useful".