Line Sorter & Deduplicator

How this works

Each output is a pipeline of small operations, and the ORDER of those operations changes the answer. That is the whole lesson of this tool. ⚠️ DE-DUPLICATION COMPARES RAW STRINGS. Two lines are duplicates only if they are byte-for-byte identical. So ' banana' and ' banana ' are NOT duplicates, and a de-duplicate pass over a list pasted out of a spreadsheet or a web page will silently miss most of them - because copied text carries trailing spaces you cannot see. THE FIX IS TO TRIM FIRST. Trimming each line, then removing the blanks, then de-duplicating, is the order that actually works. Do it in any other order and you get a worse answer that looks correct. On the sample list here, the raw de-duplication leaves SIX lines and the cleaned pass leaves FOUR - and the two extra are not a rounding difference. The raw pass keeps a blank line, and it keeps both ' Banana' and 'Banana' as separate items because one has leading spaces. There are only four distinct things in that list. ⚠️ DE-DUPLICATION IS ALSO CASE-SENSITIVE. 'Banana' and 'banana' both survive, because they are genuinely different strings. That is often what you want for names and codes, and rarely what you want for a shopping list. There is no case-insensitive de-duplicate here, and that is deliberate - collapsing case would mean choosing which capitalisation to keep, and no tool can make that choice correctly for your data. SORTING IS LOCALE-AWARE, not raw byte order. That means it puts 'apple' before 'Banana' rather than putting every capital letter before every lowercase one, which is what a naive sort does and which nobody wants. It also handles accented characters sensibly. SORTING RAW TEXT SORTS THE SPACES TOO. A line beginning with a space sorts before one that does not, and blank lines sort to the very top - so an unclean list sorts into a mess that looks like a bug. Again: trim first. NUMBERING IS APPLIED LAST, after everything else, because numbering and then sorting would sort by the numbers. The numbered output here is numbered after cleaning and sorting, which is the order that produces a usable list. WHAT THIS DOES NOT DO: it cannot sort numerically. Sorting a list of numbers as text puts 10 before 9, because it compares character by character. If you need numeric order, this is the wrong tool and a spreadsheet is the right one.

Common questions

Why does the order of operations matter?

Because de-duplication compares raw strings, so it only treats two lines as duplicates when they are byte-for-byte identical. A line ending in a space is not equal to the same line without one - and text copied out of a spreadsheet or a web page is full of trailing spaces you cannot see. Trim each line first, then drop the blanks, then de-duplicate. On the sample list here the raw pass leaves six lines and the cleaned pass leaves four - the raw one keeps a blank line and treats ' Banana' and 'Banana' as two different items, when the list only contains four distinct things. Both outputs are shown so the difference is visible rather than asserted.

Why weren't 'Banana' and 'banana' merged?

Because de-duplication is case-sensitive, and they are genuinely different strings. For names, codes and identifiers that is exactly what you want. For a shopping list it usually is not. There is deliberately no case-insensitive option here, because collapsing case means choosing which capitalisation survives - and no tool can make that choice correctly for your data. If you want them merged, lowercase the list somewhere else first and accept that you are losing the original capitalisation.

Why is 'apple' before 'Banana'?

Because the sort is locale-aware rather than raw byte order. A naive sort compares character codes, which puts every capital letter before every lowercase one - so you would get Apple, Banana, apple, banana, which nobody wants. Locale-aware sorting groups words with their own capitalisations and handles accented characters sensibly. It is the behaviour people expect from a sorted list, and it is not what a programming language gives you by default.

My list sorted into a mess.

Almost certainly untrimmed. Sorting raw text sorts the leading spaces too, so a line beginning with a space sorts before one that does not, and blank lines sort straight to the top. The result looks like the sort is broken when the input was simply unclean. Use the top output, which trims first - and if you want to see the problem rather than take it on faith, compare the raw de-duplication row against the cleaned one.

Can it sort numbers?

Not correctly, and this is the one thing worth knowing before you use it. Sorting a list of numbers as text compares character by character, so 10 sorts before 9 and 100 sorts before 20. That is correct string ordering and wrong numeric ordering. If you need numeric sorting, this is the wrong tool - paste the list into a spreadsheet, which knows the difference between a number and a string of digits.

Why is the numbering applied last?

Because numbering first and sorting afterwards would sort by the numbers, which just restores the order you started with. The numbered output here is trimmed, de-duplicated, sorted and only then numbered - which is the order that produces a list you can actually use. It is the same principle as everything else on this page: each of these operations is simple, and the entire difficulty is the sequence.

More text tools