# How to Deduplicate a Large CRM Database | Dedupely > Hundreds of thousands of records or millions? Sync only what you need, plan around sync times, test first and merge in stages, without losing control. - URL: https://dedupe.ly/knowledge-base/deduplicating-a-large-database - Section: knowledge-base - Site index: https://dedupe.ly/llms.txt - Any page: append `.md` to its URL for this plain-text form --- ## Deduplicating a Large Database By [Gaby Salinas](https://www.linkedin.com/in/gbyslns) Updated October 30, 2026 Plan a deduplication project for 480,000 records or more, from the first sync to automation. A large database, 480,000 records and up, needs the same steps as any deduplication project, done in a set order: sync only the objects you need, plan around the first sync, test before merging, merge in stages and automate last. Dedupely handles databases of millions of records, with plans up to 20 million and bulk merges of thousands of matches at once. This Playbook shows how to run it so every stage is checked before the next one starts. ### Before you start - **A plan that covers your records.** Your record limit counts every synced record across all connected CRMs. See [Dedupely Pricing](https://dedupe.ly/knowledge-base/pricing). - **Admin rights in your CRM** to connect it. See [Connect your CRM](https://dedupe.ly/knowledge-base/connect-your-crm). - **A Salesforce Sandbox** (optional), if you want to test merges outside production. ### Step 1: Sync only the objects you need A whole object syncs, so every record in an object you turn on counts toward your record limit. Before you connect, decide which objects you'll dedupe first. - **Turn on only those objects** in **CRM Settings**. You can turn others on later. - **Start with the object that causes the most problems**, usually Contacts or Companies. - **Sandbox records count too**, like any other synced record. Up to 10% over your record limit, everything keeps working for 10 days while you upgrade or turn objects off. More than 10% over, or still over after 10 days, your account becomes read-only: syncing and scans keep running, but merging stops. Dedupely never upgrades your plan or charges you for going over. See [Billing & Usage](https://dedupe.ly/knowledge-base/billing-usage). ### Step 2: Plan around the first sync The first sync of a large database takes hours, not minutes: | Records | First sync | | --- | --- | | Up to 600,000 | 1–3 hours | | Up to 1 million | 3–10 hours | | More than 1 million | 10–12 hours | After the first sync, Dedupely refreshes every 10–30 minutes and finds new records within 1–5 minutes. Syncing never merges, edits or deletes anything in your CRM, so your team can keep working while it runs. See [Syncing Records](https://dedupe.ly/knowledge-base/syncing-records). ### Step 3: Test before you merge Treat merges as permanent: at this size a wrong rule repeats thousands of times. - **Salesforce:** connect a Sandbox and run your searches and Merge Rules there first, to see how merges affect your Flows, Apex triggers and integrations. See the [Salesforce Deduplication Guide](https://dedupe.ly/knowledge-base/salesforce-deduplication-guide). - **Every CRM:** run 1–2 Single Merges, check them in your CRM, then bulk merge 1–2 pages and check again. - **Check the Merge Preview** in **View Match Details** before each new search, so you know which values survive. ### Step 4: Merge in stages Don't merge the whole database at once. Merge one search at a time, from the safest to the riskiest: - **Exact duplicates first**, on at least three fields (for example First Name, Last Name and Email on **Exact**). - **Companies and accounts**, with Company Name on **Similar Words (Any Order)** and Website on **Domain Root**. - **Looser searches last**, like name-based searches or Dedupely Intelligence, reviewed match by match. Bulk Merge can merge a whole page or **Select all X matches** at once, with no maximum. **Cancel all** stops a running bulk merge; merges already in progress finish. Matches with more than 20 records and placeholder data go to the **Excluded** tab, so they're never merged in bulk. See [Merge Safety](https://dedupe.ly/knowledge-base/merge-safety). ### How to merge thousands of HubSpot companies Duplicate companies are often the biggest job in a large HubSpot portal. To merge them at scale: - **Match on name and domain:** Company Name on **Similar Words (Any Order)**, with Ignored terms *Inc.*, *LLC*, *Ltd.*, plus Website on **Domain Root**. Add Country and City to **Visible Fields** to spot branches. - **Choose your Merge choices first.** HubSpot limits how many associations one record can have and how many times a record can be merged. Set "Too many associations to move" and "Too many past record merges in same record" before you bulk merge. Auto Merge can't ask, so it skips matches set to **Always ask**. - **Expect HubSpot's own behavior:** HubSpot usually gives the merged company a new Record ID (old IDs redirect), and both records leave active workflows. See [How to Dedupe Companies and Accounts](https://dedupe.ly/knowledge-base/how-to-dedupe-companies-and-accounts). ### Step 5: Automate last Once the backlog is merged, keep new duplicates from piling up again: - **Turn on Auto Match** on each search you've tested, so someone reviews new matches on a schedule. See [Auto Match](https://dedupe.ly/knowledge-base/auto-match). - **Move a search to Auto Merge** once it has run without false matches, with the **As soon as possible** schedule. See [Auto Merge](https://dedupe.ly/knowledge-base/auto-merge). ### Example setup A HubSpot portal with 520,000 contacts, 210,000 companies and 40,000 deals, about 770,000 records, on the Enterprise plan (1 million records). - **Day 1:** Contacts and Companies turned on. Deals stay off until contacts and companies are done. First sync: about 3–10 hours. - **Day 2:** Merge Rules set: Primary Record set to Last Updated Record; Lose Rules on Contact owner for former reps. Merge choices set to "Merge and keep the associations that fit" and "Merge by copying the data over". - **Day 2:** 2 Single Merges and 1 bulk-merged page on the exact contact search (First Name, Last Name, Email on Exact), checked in HubSpot. - **Week 1:** exact contact search bulk merged with **Select all X matches**. Then companies on Company Name (Similar Words, Ignored terms) and Website (Domain Root), page by page for the first 10 pages. - **Week 2:** name-based contact searches and Dedupely Intelligence reviewed; unclear matches sent to Manual Review. - **Week 3:** Deals turned on and deduped the same way. Auto Match on every search; Auto Merge (As soon as possible) on the exact searches. ### Common mistakes - **Syncing every object at once.** It fills your record limit and makes the first sync longer. Start with one or two objects. - **Merging during the first sync.** Wait until the sync finishes, so your searches see every record. - **Testing on one record and merging everything.** Test 1–2 Single Merges, then 1–2 pages, for every new search. - **Skipping Merge Rules.** At this size, the default winner repeats on thousands of records. Set the Primary Record and the fields that matter first. - **Turning on Auto Merge before the backlog is done.** Automate searches that have already run without false matches. ### FAQs about large databases - **What counts as a large database?** 480,000 records or more. The steps work for any size, but the order matters more as the database grows. - **Is there a limit on how many matches I can merge at once?** No. Bulk Merge can merge every match in a search with **Select all X matches**. - **Can I test in a sandbox?** Yes, in Salesforce. Sandbox records count toward your record limit. - **Does a large first sync change anything in my CRM?** No. Syncing never merges, edits or deletes records. - **What happens if I go over my record limit?** Up to 10% over, you have 10 days to upgrade or turn objects off. After that, or if you're more than 10% over, your account is read-only until you do. ### Next up [Merge Safety See how Safety Settings keep risky matches out of bulk merges and Auto Merge.](https://dedupe.ly/knowledge-base/merge-safety) ### Still stuck? Our team helps on every plan and during the free trial, by chat, email or Zoom. - Need help? [Talk to support](https://dedupe.ly/contact) - Questions about plans? [Book a Zoom with sales](https://meetings.dedupe.ly/meetings/gaby-salinas/talk-to-sales) - Reading this with a model? [Read this page as Markdown](https://dedupe.ly/knowledge-base/deduplicating-a-large-database.md) - Feeding it the whole site? [Read llms.txt](https://dedupe.ly/llms.txt) Behind the article ### Gaby Salinas Marketing Assistant at Dedupely Gaby Salinas came to CRM data from marketing and advertising, so she looks at duplicates from two sides: the match and merge logic behind them, and what they do to the people using the CRM, like the rep who calls a customer twice, the admin who gets blamed and the cleanup nobody wants to start. As Marketing Assistant at Dedupely, she follows how RevOps teams, end users, consultants and technical teams actually work with their data: the records they stop trusting, the workarounds they build and the moment they finally decide to clean up. What she finds shapes most of what Dedupely publishes. [Connect with Gaby on LinkedIn](https://www.linkedin.com/in/gbyslns) ### Get started for free 20 free trial merges and as much free support as you need to get your duplicates under control. [Talk to sales](https://meetings.dedupe.ly/meetings/gaby-salinas/talk-to-sales)[Try for free](https://v2.app.dedupe.ly/signup) [Or sign up with Google](https://v2.app.dedupe.ly/signup/google) Running CRM cleanup for clients? [See the Partner Program](https://dedupe.ly/become-a-partner) Skip to content