Dedupe a customer list without losing the right row

Why you’d do this

A customer list grows duplicates the ordinary way: the same person signs up twice, a CRM export runs twice a month and both copies get merged into one working file, or a support agent re-adds someone who was already in the sheet. The row isn’t always identical — a phone number might be missing from one copy, a name might carry different capitalization — so “remove exact duplicate rows” usually isn’t enough, and neither is guessing by eye through a few thousand rows. The two things that matter are choosing what counts as “the same customer” and knowing, afterward, that you removed the rows you meant to and kept the one with the best data.

How, with the named tool

Dedupe CSV removes duplicate rows by whichever columns you choose; Compare CSV checks the result against the original afterward.

1. Choose the file. Tap “Choose a file to dedupe” or drop your CSV, TSV, XLSX or JSON file onto the dashed area.
2. Pick the matching columns. Under “Match duplicates by”, tick the columns that actually identify a customer — usually email, or a customer ID if the list has one. Leaving every box unticked matches on the whole row instead, so two rows only count as duplicates if every column is identical.
3. Choose which copy to keep. Under Keep, pick First occurrence or Last occurrence — whichever copy is more likely to hold the up-to-date information for that customer (often the last one added, if your list is roughly chronological).
4. Remove duplicates. Tap “Remove duplicates” — the tool previews the result and shows how many rows were removed.
5. Download. Choose CSV, XLSX, JSON or TSV for the deduped list.
6. Check what was removed. Open Compare CSV, put the original file in box A and the deduped file in box B, and pick the same identifying column (email or customer ID) as the Key column. Tap Compare — the Removed count should match the number Dedupe CSV reported, and Export diff (CSV) gives you the exact rows that didn’t make it into the final list, so you can eyeball them before trusting the result.

What to watch for

The columns you tick decide what “duplicate” means — get this wrong and you either miss real duplicates or merge two different people into one. Matching on name alone can collapse two different customers who happen to share a name; matching on email or a customer ID is usually the safer choice because those are meant to be unique per person. If you leave every box unticked, only rows that are identical in every column count as duplicates, which will miss a lot of real duplicates in a list where even one field differs between copies.

Matching is on exact cell text, so stray whitespace or different capitalization means two rows of the same customer won’t be recognized as duplicates at all. "[email protected]" and "[email protected] " (with a trailing space) are different strings as far as the match is concerned, so they’ll both survive as separate rows. If your list has inconsistent casing or spacing in the columns you’re matching on, that’s worth fixing first — a spreadsheet’s own lowercase/trim formulas, or manual cleanup, since Dedupe CSV doesn’t normalize text before comparing it.

“First occurrence” and “Last occurrence” only refer to the row’s position in the file, not which copy has better data. If the more complete or more recent record happens to be the first row for that customer rather than the last, keeping “Last occurrence” would throw away the better copy. Sort your file the way you want before deduping if row order matters to which copy survives.

FAQ

Which columns should I match duplicates on for a customer list?

Whatever uniquely identifies a person in your data — email address or a customer ID are the usual choices. Matching on name alone risks collapsing two different customers who share a name into one row; matching on the whole row (leaving every box unticked) will miss duplicates where even one field, like a phone number, differs between copies.

How do I know which rows were actually removed?

Run the original file and the deduped file through Compare CSV, using the same identifying column as the key. The Removed count should match what Dedupe CSV reported, and exporting the diff gives you the exact rows that were dropped, so you can check them before treating the dedupe as final.

Why didn’t Dedupe CSV catch two rows I know are the same customer?

Matching is on exact cell text — a difference in spacing, capitalization, or a value present in one row and missing in the other (like a phone number filled in on one copy only) means the two rows won’t be recognized as duplicates, even though they’re the same person. Clean up inconsistent spacing or case in the matching columns before deduping if this happens often.