Duplicate Line Remover

How to remove duplicate lines from a list

Paste it on the left. The unique lines appear on the right immediately, and below them a panel names every line that repeated with a count.

Nothing to set. Comparison ignores capitals and surrounding spaces, which is what people mean when they call two lines the same.

Why the report matters more than the list

Because a shorter list does not tell you what happened.

You paste 400 addresses, get 340 back, and have no way to know whether that is right. Sixty duplicates could be one address repeated sixty times, which usually means a broken export, or sixty separate pairs, which usually means two lists were merged. Those are different problems.

The count per line answers it. An entry appearing eleven times is often the whole finding, and it is the part most dedupe tools throw away.

What counts as the same line

Capitals are ignored. Ana@acme.com and ana@acme.com reach the same inbox, and a tool that keeps both has not really deduped anything.

Leading and trailing spaces are ignored too, since a list assembled by copying from three places will have padding on some entries and not others.

The line that survives is the first one, kept exactly as you wrote it. Only the comparison is relaxed, never the output, so a list where capitalisation is deliberate comes back untouched.

Why two identical lines sometimes refuse to match

A carriage return, almost always.

Text from Windows ends each line with two characters rather than one. A tool that splits on the newline alone leaves the return sitting at the end of every line, invisible, and two lines that look identical are compared as different. Line endings are normalised here before anything else happens, which is why those match.

If lines still refuse to match after that, something invisible is inside them rather than at the end, and the invisible character detector will show it with a line and column.

Blank lines are left alone

They pass straight through. A blank line separates things rather than being one of them, so treating it as a duplicate would quietly merge two paragraphs into one.

If the list also has padding and stray spacing you want gone, run it through the whitespace cleaner first. That combination turns a list assembled from three different sources into something you can actually import.

Everything runs in your browser. Nothing is uploaded.