Dedup belongs at the write path, not after

Duplicates are treated as a cleanup problem. A team notices the same contact twice, runs a dedup tool, merges the records, and considers it solved. Three weeks later the duplicates are back, because the dedup ran on the symptom and not the cause. The cause is that duplicates entered the system, and nothing stopped them at the door.

Dedup is a write-path problem. It belongs at the moment a record is created or updated, not after it has been enriched, contacted, and counted in a report. By the time a duplicate is sitting in the CRM, the damage is already done: the enrichment credits were spent twice, the contact may have been emailed twice, and the merge has to reconcile two histories that were never supposed to exist. Post-hoc dedup is a refund you will never get.

The fix is to match at entry, on the identifiers you actually hold. Every record carries some set of stable keys: a company domain, a LinkedIn URL, a provider’s primary key, an email address. The write path checks the incoming record against those keys before it persists anything. If a match exists, the incoming record updates the existing one, or gets suppressed, depending on the rule. If no match exists, it’s new. The duplicate never gets created, so it never has to be cleaned up.

The hard case is the company that defeats matching. A company with two domains and two LinkedIn pages can’t be resolved by a simple key match, because it looks like two companies, and the expensive fix, full entity resolution, is rarely worth it. The practical backstop is email-level dedup at the send stage. The company match may miss, but the person match on email usually doesn’t, so the same contact doesn’t get emailed twice even when their employer reads as two accounts. It’s not perfect, but it catches the failure that costs the most: the duplicate send that a prospect notices and that marks you as sloppy.

Most teams skip write-path dedup because it’s invisible. A dedup tool that runs on Tuesday produces a report you can point at. A write-path constraint that prevents the duplicate on Monday produces nothing to point at, because the thing it prevented never happened. Invisibility is the sign it’s working. The teams with clean data aren’t running better dedup tools. They’re running dedup earlier, at the only point where it actually prevents waste.

Run the match where the record enters, suppress before the credit gets spent, and dedup the send on email when the company key fails. The duplicates stop recurring, and the cleanup tool becomes a periodic audit instead of a weekly firefight. Dedup isn’t a job you do after. It’s a constraint you build in.

Dedup after enrichment is a refund you’ll never get. I build the constraint at the write path. Worth a 30-minute look at your stack? → Book a call.