How to Sort and Deduplicate a List

De-duplication compares raw strings, so 'apple' and 'apple ' both survive. Trim first — or you get a worse answer that looks correct.

The same list, two orders of operations

Paste this eight-line list — the kind you get from a spreadsheet, with invisible trailing spaces and stray blanks:

  Banana
 apple

  banana 
cherry

 apple
Banana

Approach	Result
De-duplicate straight away	6 lines
Trim → drop blanks → de-duplicate	4 lines

There are only four distinct things in that list.

The raw pass keeps a blank line, and it keeps both Banana and Banana as separate items — because one has leading spaces. It looks like it worked.

Your list, one item per line

This sample deliberately has trailing spaces, blank lines and mixed case — so you can see what each step catches.

48 / 20,000 characters

Cleaned, de-duplicated and sorted

Trim, drop blanks, de-duplicate, then sort. This is the order that works.

apple
banana
Banana
cherry

— and numbered

Numbered LAST. Numbering before sorting would sort by the numbers.

1. apple
2. banana
3. Banana
4. cherry

De-duplicated WITHOUT trimming first

Compare against the top row. This is what raw de-duplication misses, and it is why order matters.

  Banana
 apple

  banana 
cherry
Banana

Cleaned but not sorted (original order kept)

Use this when the order of the list means something.

Banana
apple
banana
cherry

Lines you pasted

After blanks are dropped — blank lines are not items.

Lines after de-duplication

The difference between these two is how many duplicates you had.

Open the Line Sorter & Deduplicator on its own page to bookmark or share it.

Why it happens

De-duplication compares raw strings. Two lines are duplicates only when they're byte-for-byte identical.

Text copied out of a spreadsheet, a PDF or a web page is full of trailing spaces you cannot see. So the duplicates are still there, the count looks plausible, and nothing tells you it went wrong.

The order that works:

Trim each line
Remove blank lines
De-duplicate
Sort
Number — last

Each step is trivial. The entire difficulty is the sequence.

Number last, or you sort by the numbers

Obvious once stated, and easy to do backwards.

Numbering first and sorting afterwards sorts 1., 10., 11., 2. — restoring roughly the order you started with, in a way that looks deliberate.

The tool's "— and numbered" output trims, drops blanks, de-duplicates, sorts, and only then numbers — which is the order that produces a list you can hand to someone.

What each output is for

Four treatments, because the right one depends on what you're doing:

Cleaned, de-duplicated and sorted — the default answer, for a list where order carries no meaning
— and numbered — the same thing, ready to paste into a document
Cleaned but not sorted — when the original order is information: a sequence of steps, a ranking, a chronology. De-duplicating without sorting is a genuinely different operation and easy to forget exists
De-duplicated without trimming — there purely so you can see what the naive approach misses

De-duplication is case-sensitive

Banana and banana both survive, because they're genuinely different strings.

For names, codes and identifiers that's exactly right. For a shopping list it usually isn't.

There's deliberately no case-insensitive option here — collapsing case means choosing which capitalisation survives, and no tool can make that choice correctly for your data. If you want them merged, lowercase the list first and accept you're losing the original capitalisation.

Sorting is locale-aware, not byte order

apple comes before Banana.

A naive sort compares character codes, which puts every capital letter before every lowercase one — giving you Apple, Banana, apple, banana. Nobody wants that.

Locale-aware sorting groups words with their own capitalisations and handles accented characters sensibly. It's the behaviour people expect from a sorted list, and notably not what most programming languages give you by default.

"My list sorted into a mess"

Almost always untrimmed.

Sorting raw text sorts the leading spaces too, so a line beginning with a space sorts before one that doesn't — and blank lines sort straight to the top. The result looks like the sort is broken when the input was simply unclean.

Same fix as everything else on this page.

It cannot sort numbers

Worth knowing before you use it for the wrong job.

Sorting a list of numbers as text compares character by character, so 10 sorts before 9, and 100 before 20.

That's correct string ordering and wrong numeric ordering. If you need numeric sorting, paste the list into a spreadsheet — which knows the difference between a number and a string of digits.

Keep the original

De-duplication isn't reversible. Once two identical lines have become one, nothing records that there were two.

If the count of duplicates was itself information — how many times something appeared, how many people picked the same option — you needed that before you cleaned the list, not after.

That's why the tool shows the line count before and after, side by side. The difference is often the answer you actually wanted.

Print the cleaning reference — the order of operations, and a checklist for before you trust a de-duplicated list.