Remove duplicate lines

Duplicate lines show up in the boring places: an emailed list of attendees pasted twice, a SKU export that joined two warehouses, a log where the same error printed in a loop, a column copied out of Excel with a header repeated. People still open a spreadsheet, click Data → Remove Duplicates, and hope the workbook was not already a mess. This tool is that button without Excel and without an upload. Paste or drop a list, get it back with repeats gone.

“Remove duplicates” can mean three different jobs, so there is a mode toggle. Keep first occurrence is the usual request: the first time a line appears, it stays; later copies drop. Show only the duplicates is for auditing — you want to see what was repeated, not the clean list. Show only lines that appear exactly once is the inverse: leftovers after you throw out anything that showed up twice. Stats under the box report lines in, unique keys, and how many copies were removed in keep-first mode.

Options exist because real lists are dirty. Case insensitive treats Apple and apple as the same key. Trim whitespace so a trailing space does not keep a “different” line. Ignore blank lines so empty rows do not count as a value. Sort is off by default (original order, first-seen). A → Z and Z → A are there for when you want a directory. Sorting uses the comparison key, so case-insensitive sort does not surprise you with ASCII.

This is a heavily searched utility with weak incumbents that still POST the list. If the list is emails, that is a problem. If the list is patient IDs, that is a bigger problem. imissfiles never receives it. Stay under about 5 MB. A few hundred thousand short lines is usually fine in a modern tab; a 20 MB dump should be split first so the worker does not grind.

Typical path: extract a column with PDF to Text or Image to Text, paste it here, download the unique list, then use Find and Replace if a domain suffix is wrong. Word Counter will tell you how many tokens remain. None of those steps leave this device. Do not paste a spreadsheet formula — this is line identity, not a fuzzy match. “Smith, J” and “J. Smith” stay two rows. If you need a fuzzy merge, that is a different (and still local, someday) tool. For now, trim and case-insensitive compare is the honest feature set.