Keyword and tag lists
Collapse repeated entries while keeping the first editorial ordering intact.
Enable Ignore case and Trim whitespace; leave Sort A–Z off if priority lives in the source order.
Free browser tool
Text & Writing
Remove Duplicate Lines
Strip repeated lines from lists, logs, keywords, or exports — with options to ignore case, trim whitespace, drop empty lines, and sort A–Z.
- Free public tool
- No sign-up for this tool
- Processing clearly labeled
- Instant results
Content last reviewed
0 lines
Overview
Keep the first matching line and make the comparison rules explicit
Paste a line-based list, decide whether case and edge whitespace matter, then keep the first occurrence of each comparison key. Sorting and blank-line removal are separate choices, so cleanup does not silently reorder the source.
Stable by default
The first occurrence stays where it appeared; later matches are removed without moving the surviving lines.
Case is your choice
Ignore case lowercases only the comparison key. With it off, Apple and apple remain different lines.
Whitespace rules are visible
Trim whitespace compares without leading and trailing spaces while preserving the original spelling and spacing of the kept line.
Copy or download
The live result can be copied or saved as a UTF-8 text file named deduplicated.txt.
How it works
Using Remove Duplicate Lines, start to finish
Three steps, in the order the tool above actually takes them.
- 1
Paste one item per line
Use the example or paste a column, log excerpt, URL list or other newline-separated text.
- 2
Define a match
Choose case handling, edge trimming and empty-line removal before deciding whether the survivors should be sorted.
- 3
Review the counts
Compare Lines in, Unique kept and Removed, then copy or download the live result.
Use cases
What people bring here
The jobs this page is usually opened for, and the setting that makes each one quick.
A copied spreadsheet column
Clean a single column pasted from Excel or Google Sheets without uploading the workbook.
Turn on Remove empty lines when blank cells should disappear from the exported list.
URL inventories
Remove exact repeated addresses before a crawl, redirect review or content migration.
Keep Ignore case off: path capitalization can be meaningful on a case-sensitive server.
Email recipient cleanup
Reduce an address-per-line list before moving it into a mailing workflow you are authorized to use.
Ignore case usually fits addresses, but this tool does not validate mailboxes or consent.
Repeated log messages
Get a compact set of exact messages for triage without claiming similar-looking errors are the same.
Leave Trim whitespace on; timestamps and request IDs still make otherwise similar lines distinct.
Alphabetized reference lists
Produce one copy of each value and sort the survivors for a glossary or checklist.
Enable Sort A–Z only after deduplication; it uses a locale-aware, case-insensitive comparison.
Exact means exact
What this page considers the same line
Deduplication is only predictable when the comparison key is predictable. These switches change the key, not the original line that is kept.
| Difference | Without the matching option | With the matching option |
|---|---|---|
| Apple / apple | They remain two lines because case differs. | Ignore case maps both comparison keys to apple, so the first one survives. |
| Spaces at the edges | With trimming off, item and item followed by a space are distinct. | Trim whitespace ignores leading and trailing spaces for matching while retaining the first line's original spacing. |
| Spaces inside a line | Two interior spaces are never normalized by this tool. | Even with Trim whitespace on, foo bar and foo bar remain different. |
| Blank lines | With Remove empty lines off, the first blank comparison key can remain. | Remove empty lines drops every line whose comparison key is empty. |
| Typos and punctuation | Colour and color, or item and item., are separate values. | There is no fuzzy or semantic matching; those differences require a manual review or another tool. |
Apple / apple
They remain two lines because case differs.
Ignore case maps both comparison keys to apple, so the first one survives.
Spaces at the edges
With trimming off, item and item followed by a space are distinct.
Trim whitespace ignores leading and trailing spaces for matching while retaining the first line's original spacing.
Spaces inside a line
Two interior spaces are never normalized by this tool.
Even with Trim whitespace on, foo bar and foo bar remain different.
Blank lines
With Remove empty lines off, the first blank comparison key can remain.
Remove empty lines drops every line whose comparison key is empty.
Typos and punctuation
Colour and color, or item and item., are separate values.
There is no fuzzy or semantic matching; those differences require a manual review or another tool.
Privacy
Where your work is processed
The list is split, compared, sorted and downloaded in this tab. The text itself is not uploaded or autosaved; only the split-panel width can persist for the current tab session.
Your text is never sent
Every result on these pages is computed by JavaScript in this tab as you type. There is no submit step and no request that carries a word of what you entered.
Your content is not autosaved
No source text, generated password or result is written to localStorage or sessionStorage. Split-view tools remember only the divider position for the current tab session; reloading clears the work itself.
The page records one aggregate view
The page shell records that this tool was opened, sending its slug and nothing from the form. It is skipped when your browser signals Do Not Track or Global Privacy Control.
One caveat on that
The endpoint behind the counter is rate-limited like any other, which briefly records a requesting address to enforce the limit. It is never joined to anything you typed, because none of that is sent.
More detail in how processing works and our privacy policy.
Limits
Comparison, ordering, and output
The values this tool actually enforces, not a rounded-up version.
- Unit
- One line
- Input is split at newline characters; embedded commas and tabs stay part of the line.
- Default trim
- On
- Leading and trailing whitespace is ignored for the comparison key.
- Default case
- Sensitive
- Ignore case starts off, so capitalization differences survive.
- Default order
- First seen
- Only later matches are removed and the remaining sequence is stable.
- Blank handling
- Optional
- When blanks are kept, duplicate blank keys still collapse to the first one.
- Sort
- A–Z, base sensitivity
- localeCompare orders survivors without treating letter case as a primary difference.
- Near-duplicates
- Not detected
- No edit distance, stemming, punctuation folding or semantic comparison runs.
- Export
- deduplicated.txt
- The current result is assembled into a local UTF-8 text download.
Near-duplicates and natural-number sorting are outside this pass
Comparison is exact after the optional trim and lowercase steps — the tool does not detect near-duplicates, so misspellings, extra punctuation, or different word orders count as distinct lines. Only leading and trailing whitespace is ignored; spacing differences inside a line make lines unique. Sorting is alphabetical only (no numeric or natural sort), so line 10 sorts before line 2. Very large lists are processed in memory in your browser; hundreds of thousands of lines work on a typical machine, but extremely large files may be slow.
FAQ
Line deduplication questions
The ones that actually come up.
Guides