Data deduplication
In short
Data deduplication is the process of identifying records that refer to the same real-world company or person and merging them into one. In a CRM it applies to leads, contacts, accounts, and sometimes deals.
Duplicates split activity history, inflate counts, and cause two reps to contact the same buyer. Deduplication restores a single source of truth for each entity.
How duplicates appear
- Multiple entry points: Forms, imports, integrations, and manual entry each create records.
- Format differences: "Acme Inc.", "ACME" and "Acme, Incorporated" look different to a system.
- Lead vs. contact splits: The same person exists as a lead and as a contact on an account.
- Subsidiaries and domains: A parent and its regional entities get logged as unrelated accounts.
How to deduplicate
1. Normalize first
Standardize case, punctuation, legal suffixes, and domains so matching has clean inputs.
2. Choose match keys
For accounts, a website domain is usually the strongest key. For contacts, use email, then name plus company. Fuzzy matching helps catch near-misses.
3. Set merge rules
Decide which record survives and which field value wins, for example the most recently verified value or the value from the most trusted source.
4. Review borderline matches
Auto-merge high-confidence pairs and send uncertain ones to a person.
Example
Say a CRM has three records: "Northwind Ltd" with the domain northwind.com, "Northwind Limited" with no domain, and a lead named after the same company. After normalizing names and matching on domain, they merge into one account. Activity from all three is combined, and the owner is set by the territory rules.
Prevent duplicates, not just remove them
Add duplicate checks at form submission and import, enforce domain as a required account field, and match inbound leads to existing accounts before creating new ones. Cleanup without prevention returns within months.
Common mistakes
- Merging without backups: Merges are hard to undo. Export or snapshot first.
- Matching on name only: Common names create false merges.
- Ignoring relationships: Merging a subsidiary into its parent can break territories and reporting.
Data deduplication FAQs
What is data deduplication in a CRM?
Data deduplication in a CRM is finding records that represent the same company or person and merging them into one. It keeps activity history together, makes counts accurate, and stops reps from working the same buyer twice.
How do you find duplicate records in a CRM?
Standardize names, domains, and emails first, then match on strong keys such as website domain for accounts and email for contacts. Add fuzzy matching for near-misses and send low-confidence pairs to a person for review.
What is the difference between data deduplication and data cleansing?
Deduplication removes or merges duplicate records. Data cleansing is broader and also fixes errors, fills gaps, and removes invalid values. Deduplication is one step within a cleansing effort.
How do you prevent duplicate leads and accounts?
Check for matches at form submission and import, require a domain on every account, match new leads to existing accounts before creating records, and standardize fields so variations of the same name are caught.
