# Spreadsheet Deduplication – Definition, FAQs & How Dedupely Helps > Spreadsheet deduplication explained: why remove-duplicates functions miss near-duplicates, how to dedupe a CSV, and when to merge rows instead of deleting. - URL: https://dedupe.ly/glossary/spreadsheet-deduplication - Section: glossary - Site index: https://dedupe.ly/llms.txt - Any page: append `.md` to its URL for this plain-text form --- [Glossary](https://dedupe.ly/glossary) Spreadsheet Deduplication ## Spreadsheet Deduplication Spreadsheet deduplication is the process of finding and resolving duplicate rows in a spreadsheet or CSV file, so each person, company or item appears once. It's used to clean contact lists, event sign-ups, exports from other systems and files combined from several sources, often before they're imported into a CRM or used for a mailing. Duplicate rows come in two kinds. Exact duplicates hold the same values in the columns being compared, and the remove-duplicates function built into a spreadsheet can find them. Near-duplicates describe the same record with small differences, such as a typo, a nickname, an extra space or a phone number in another format, and need looser matching to find. Once duplicates are found, they're resolved in one of two ways: deleted, so one row stays as it is, or merged, so the surviving row keeps the best value from each. See how Dedupely dedupes spreadsheets before they reach your CRM [Try for free](https://v2.app.dedupe.ly/signup) [Talk to sales](https://meetings.dedupe.ly/meetings/gaby-salinas/talk-to-sales) Overview Spreadsheet deduplication finds the rows in a spreadsheet or CSV that describe the same person, company or item, and reduces them to one. Exact repeats are simple to remove. Near-duplicates need looser matching, and merging them keeps values that deleting a row would lose. On this page - What Is Spreadsheet Deduplication? - How Spreadsheet Deduplication Works - Why Spreadsheet Deduplication Matters for CRM Teams - Getting Started With Spreadsheet Deduplication - Key Takeaways: Spreadsheet Deduplication - Frequently Asked Questions About Spreadsheet Deduplication - Related Terms and Concepts See how Dedupely dedupes spreadsheets before they reach your CRM [Try for free](https://v2.app.dedupe.ly/signup) [Talk to sales](https://meetings.dedupe.ly/meetings/gaby-salinas/talk-to-sales) ### What Is Spreadsheet Deduplication? Spreadsheet deduplication, also called CSV deduplication, covers any tabular file: a spreadsheet, a CSV exported from a CRM, or a list from an event or a partner. The rows can describe any object, such as contacts and companies from HubSpot, leads and accounts from Salesforce, or people and organizations from Pipedrive. A simple example: a contact list has three rows for one person. Row 1 is "J. Smith, jsmith@acme.com, (555) 010-2000". Row 2 is "John Smith, jsmith@acme.com, blank phone, Sales Director". Row 3 is "Jon Smith , john.smith@acme.com, 555-010-2000". Removing duplicate rows on the email column deletes row 2 and keeps row 1, which loses the job title. Row 3 stays, because its email is different, even though it's the same person. There are three common approaches, and they're often combined. A built-in remove-duplicates function deletes rows that match exactly on the columns you pick. Its strength is speed: it's already in the spreadsheet and clears exact repeats in one step. Formulas, highlighting and sorting flag likely duplicates so someone can check and fix them by hand. Their strength is control, since each change is a decision someone made. A matching tool compares rows on several columns with looser methods and merges the matches. Its strength is that it finds near-duplicates and keeps the values from every row it merges. Resources: - [What is a duplicate?](https://dedupe.ly/blog/what-is-a-duplicate) ### How Spreadsheet Deduplication Works Whatever the tool, deduplicating a file follows the same steps, from the raw rows to a clean file: - **Bring the rows together.** Files from several sources are combined into one, with the same column headers, so a person who appears in two lists can be compared with themselves. - **Even out the values.** Extra spaces, mixed capitalization and phone numbers in different formats are made consistent, because a strict comparison treats "Acme" and "Acme " as different values. - **Choose the columns to compare.** Columns that point to one person or company, such as email address or website, make the strongest keys. A column many rows share, such as a city or a first name, pairs rows that only happen to agree. - **Choose how each column is compared.** Exact comparison suits emails and IDs. Looser methods catch typos, names that sound alike or emails from the same domain. Combining a few columns keeps one shared value from deciding a match alone. - **Group and check the matches.** Rows that match are grouped into duplicate sets. Placeholder values, such as "N/A" or test@test.com, can tie unrelated rows together, so a sample is checked first. - **Resolve each set.** Deleting keeps one row as it stands. Merging builds one row from the set, choosing which value to keep in each column. - **Save or export the clean file.** The result is a file with one row per record, ready to import or send. Resources: - [Match Options](https://dedupe.ly/knowledge-base/match-options) (KB) ### Why Spreadsheet Deduplication Matters for CRM Teams Many lists reach a CRM as a spreadsheet, so a duplicate in the file often becomes a duplicate record. Every repeat that isn't caught in the file has to be found later in the CRM, after it has an owner, activity and links to deals or companies. How duplicates are resolved matters as much as finding them. Removing duplicates on one column often keeps the first row and drops the rest, so a phone number, a job title or a newer address in the deleted row is gone. Merging keeps those values. Choosing the wrong column causes the opposite problem: removing duplicates on name alone deletes rows for different people who share a name, and that data isn't in the file anymore. Near-duplicates that stay in the file have their own cost. A mailing sent from the list reaches the same person twice, event counts and campaign lists overstate the real audience, and a list split across sales reps can put one contact with two of them. When the file is imported, tools that bill by contact or record count charge for each duplicate it adds. Spreadsheets also shape what gets merged later. A clean file with consistent formats gives any import check, and any later duplicate search, values that compare cleanly. Resources: - [Why your CRM database is turning into a mess (and how to fix it)](https://dedupe.ly/blog/why-your-crm-database-is-turning-into-a-mess) - [Should You Clean Spreadsheets Before Uploading or Clean Your CRM After?](https://dedupe.ly/blog/clean-spreadsheets-before-uploading-or-clean-your-crm-after) ### Getting Started With Spreadsheet Deduplication Start by keeping the original file unchanged, so any row that's deleted or merged can still be checked later. Then combine the files that belong together under one set of column headers, and even out spaces, capitalization and phone formats before comparing anything. Clear the exact repeats first, on a strong column such as email address, and check what each pass removes before moving on. Then look for the near-duplicates that are left, comparing a few columns together and using looser comparison on names and phone numbers. Check a sample of each set, watching for placeholder values that tie unrelated rows together. When rows hold different information, merge them instead of deleting them, so the surviving row keeps the email from one, the phone number from another and the job title from a third. Decide in advance which row's values win when two rows disagree, such as the most complete or the most recent. Then export the result and keep the clean file as the version to import or share. In Dedupely, you upload a spreadsheet as a CSV, with no CRM needed, and a Duplicate Search matches its rows against each other with the same Match Options used for CRM records. You merge the duplicate sets it finds and export a clean CSV, and several files can go into one upload when their headers match. Resources: - [Merge duplicates in spreadsheets](https://dedupe.ly/knowledge-base/connect-csv) (KB) - [Duplicate Searches](https://dedupe.ly/knowledge-base/search-pads) (KB) ### Key Takeaways: Spreadsheet Deduplication Spreadsheet deduplication finds the rows in a spreadsheet or CSV that describe the same person, company or item, and reduces them to one. A built-in remove-duplicates function clears exact repeats, but near-duplicates need looser matching on a few columns, and merging keeps values that deleting a row would lose. Evening out formats first, checking a sample and keeping the original file make the result safer. In Dedupely, you can upload a CSV with no CRM, match it against itself with the same Match Options used for CRM records, merge the duplicates and export a clean CSV. CSV records don't count toward the record limit, and once the clean file is imported, Duplicate Searches on the CRM find overlaps with records already there. Resources: - [Dedupely product overview](https://dedupe.ly/product) ### Frequently Asked Questions About Spreadsheet Deduplication #### Why doesn't a spreadsheet's remove-duplicates function catch near-duplicates? Because it compares values exactly, so two rows count as duplicates only when every chosen column holds the same value. A typo, a nickname, a trailing space, a different email or a phone number written with dashes instead of parentheses all make rows look different, even when they describe the same person. Depending on the spreadsheet, capitalization may count as a difference too. The function also deletes rather than merges: it often keeps the first row in each set and removes the rest, along with any values only those rows held. Evening out spaces and formats first helps it find more, but it still can't pair "Jon" with "John" or two emails from the same company. In Dedupely, a Duplicate Search on a CSV gives each column its own Match Option. Similar catches minor typos and strips formatting from phone numbers, Fuzzy pairs names that sound alike, such as Sean and Shawn, and Domain Root matches emails from the same domain. #### How do you dedupe a contact list before importing it into a CRM? Combine the files, even out the formats, match the rows on a few key columns and merge the duplicates, so each contact appears once in the file you import. Start with columns that point to one person, such as email address, then add name and company with looser comparison to catch typos and second emails. Remove placeholder values, such as "N/A" or test@test.com, which can pair unrelated rows. When two rows hold different details, merge them so the job title from one and the phone number from the other both survive. Deduping the list only finds repeats inside the file, so the contacts already in the CRM still need a check after the import. In Dedupely, you upload the list as a CSV, with no CRM needed, and a Duplicate Search finds its duplicate sets. You merge them and export a clean CSV to import. #### How large a CSV file can you dedupe? It depends on the tool, and on how the rows are compared. Spreadsheet apps cap the number of rows a sheet can hold, and formulas that compare every row with every other row slow down as a file grows. Exact comparison on one column scales well, while looser comparison across several columns takes more work per row. For a very large list, splitting it by a column such as country can make it easier to handle, though duplicates that cross the split won't be compared. In Dedupely, one CSV upload can hold up to 10 files, 100 MB in total, 1,000,000 rows and 50 columns. Each cell can hold up to 350 characters, and each line up to 4 MB. CSV records don't count toward the record limit, so deduping a file doesn't use any of the records you sync from a CRM. #### How does spreadsheet deduplication work across several files? The files are combined into one with the same column headers, and then the combined file is deduped, so a person who appears in two lists is compared with themselves. Before combining, check that each column means the same thing in every file: one list's "Name" may hold a full name while another splits first and last name, and phone numbers or country names may follow different formats. Rename and reorder columns so they line up, and add a column that records which file each row came from, so you can trace a value back after merging. Then match on a few key columns and merge the duplicate sets, keeping the most complete or most recent value in each column. In Dedupely, a CSV upload can combine several files into one when their headers match, up to 10 files per upload. The combined rows are matched against each other in a Duplicate Search, merged and exported as one clean CSV. #### How do you check a CSV against records already in your CRM? Import the cleaned file and run a duplicate search across the CRM afterward, or compare the file with an export of the CRM's records before importing it. The first way uses the CRM's own data as it stands, so it also catches overlaps on fields the import doesn't check, such as name plus company or phone number. The second way keeps overlaps out of the CRM entirely, but it needs a current export, and the export can be out of date by the time the file is imported. Either way, a CRM's import check often looks at one field, such as email for contacts, so a row with a different email can still come in as a new record. In Dedupely, a CSV is matched only against itself, not against CRM records. Dedupe the file as a CSV, export it clean and import it, then run a Duplicate Search on the CRM once Dedupely detects the new records, within 1–5 minutes. ### Related Terms and Concepts #### [CRM import](https://dedupe.ly/glossary/crm-import) A CRM import brings a file's rows into a CRM in bulk. Spreadsheet deduplication is the step before it, so the file's own duplicates don't become duplicate records. #### [Exact matching](https://dedupe.ly/glossary/exact-matching) Exact matching is how a spreadsheet's remove-duplicates function compares rows. It clears exact repeats in a file but leaves the near-duplicates behind. #### [Similar matching](https://dedupe.ly/glossary/similar-matching) Similar matching catches typos and phone numbers in different formats. It finds the rows in a file that a remove-duplicates function treats as different. #### [Multi-field matching](https://dedupe.ly/glossary/multi-field-matching) Multi-field matching compares rows on several columns at once. It keeps a shared value in one column, such as a common last name, from pairing different people in a list. #### [Deduplicating data with AI](https://dedupe.ly/glossary/deduplicating-data-with-ai) Some teams paste a spreadsheet into an AI assistant to find duplicate rows. It's another way to dedupe a file, with its own limits on file size and on checking what was changed. #### [Record limit](https://dedupe.ly/glossary/record-limit) A record limit counts the records a tool syncs from a CRM. Records in an uploaded spreadsheet sit outside that count in some tools, which matters when deduping large files. Skip to content