How it works
The tool reads your text line by line and remembers every line it has seen. When a line matches one seen earlier, it is dropped. Order is preserved, so the result is your original list with the repeats removed. It is not sorted unless you sort it afterwards.
By default, spaces at the start and end of a line are ignored when comparing, because stray whitespace is the most common reason “identical” lines fail to match. The line that is kept is left exactly as it was.
Rules this tool follows
- Case sensitive (on by default): “Apple” and “apple” are different. Turn it off to treat them as duplicates.
- Ignore surrounding spaces (on by default): “ apple” and “apple ” match.
- Keep all empty lines (on by default) leaves blank lines in place so paragraph spacing survives. Turn it off to keep only the first blank line.
Example
Input
banana apple cherry apple Banana cherry
Output
banana apple cherry Banana
Common uses
- Cleaning email lists, keyword lists and URL lists.
- Removing repeated rows before pasting into a spreadsheet.
- Deduplicating log lines or error messages.
Limitations and notes
- Only whole lines are compared. To find repeated words inside a text, use the Word Frequency Counter.
- Lines that differ only in invisible characters, such as a zero-width space, are treated as different.
Questions
Does it sort the lines?
No. The original order is kept. Use Sort Lines afterwards if you want an alphabetical list.
Which copy is kept?
The first one. Later copies are removed.