Blog
Duplicates

How AI duplicate matching works and what to ask a vendor

7
min read
An AI matching machine scoring two records 0.92 and 0.64 while a person wonders

Table of contents

Talk to sales
Try for free
View as Markdown

TL;DR

  • AI duplicate matching uses AI to judge records, score pairs or design matching logic.
  • Where AI runs decides whether results repeat and where CRM data travels.
  • A good AI matching tool shows why records matched and keeps whole groups together.
  • AI finds candidates, but people still decide what a duplicate is and what survives.
  • Dedupely Intelligence is AI-powered matching that runs as fixed code on Dedupely's infrastructure.

More and more deduplication tools describe their matching as AI-powered. The label sounds like one thing, but it covers very different designs.

In one tool, a language model reads your records every time it looks for duplicates. In another, a model learned what duplicates look like and scores each pair.

In a third, AI helped write the matching logic, and the app runs plain code. Each design behaves differently with your data, your results and your merges.

In this post, you'll see how each design works and what it means for consistency, privacy and control. You'll also get a list of questions to ask before you trust an AI deduplication tool with your CRM.

Why AI is used in duplicate matching

Classic matching compares fields with rules you set, like exact email or similar name. It works well once you know which fields to combine and how strictly. Our explainer on how duplicate matching works covers those match types.

The hard part is knowing the combinations. Complex duplicates only show up when several fields line up, and every database overlaps differently. AI is useful there: it can look across many patterns and find combinations a person wouldn't think to try.

Three ways AI can work in duplicate matching

1. A model judges your records at run time

The tool sends records, or pairs of records, to a language model and asks whether they're the same. The model answers each time the matching runs.

This handles messy text well, like nicknames or company names written five ways. The trade-off is that a language model predicts likely answers, so the same records can get a different answer on another run. Your CRM data also travels to wherever the model runs.

2. A trained model scores each pair

Some tools train a model on examples of duplicates and non-duplicates. In use, it gives each pair a score, and pairs above a threshold become matches.

Scores are usually consistent for the same model. The model can also change as it's retrained, sometimes on decisions users make. What it learned can be hard to explain, so a score of 0.92 doesn't tell you which fields drove the match.

3. AI designs the logic, and fixed code runs it

Here AI works earlier, while the matching logic is being built. It helps find the field combinations and comparisons that catch complex duplicates. What runs on your data is ordinary code that gives the same result for the same records every time.

This keeps results repeatable and keeps a model out of the path of your data. The trade-off is that the logic only changes when the vendor improves it.

The three designs at a glance

Model judges at run time Trained model scores pairs AI-designed logic, fixed code
Same input, same result Not guaranteed Usually, until retrained Yes
Where CRM data goes To wherever the model runs Depends on where the model is hosted Stays where the code runs
Why records matched A text explanation, if asked A score, often without field-level reasons The fields and comparisons that agreed
Learns from your decisions Only through prompts Often, if retrained on feedback No; changes come from the vendor

Tools can also mix these designs, so ask which one runs on your data.

What AI doesn't decide for you

Whatever the design, AI finds candidates. Two decisions stay with your team.

The first is what counts as a duplicate. That's case by case: two branches can be one customer for one company and two for another. No model knows how your business sells and bills, so someone has to confirm or reject what it finds.

The second is what the merged record keeps. Which record survives, who owns it and which email wins are business rules. They should be set and previewed before anything merges, because a merge changes your live CRM.

Questions to ask an AI deduplication tool

These questions cut through the label and show how a tool behaves with your data.

Question Why it matters
Where does AI run: while building the logic, or on my records? It decides whether results repeat and whether data leaves the tool
Is my CRM data sent to an outside AI model or used to train one? Your security team will ask, and many contracts limit it
Will the same records always give the same result? You can't test a search you can't repeat
Can I see why two records matched? Reviewers need the fields that agreed to decide quickly
Does it show groups of three or more records together? Duplicates often come in groups, and pairs split the decision
What happens when I reject a match? It should stay rejected, and you should know if it changes the model
Who decides which values survive the merge? Your own rules and a preview should decide what the merged record keeps
Can it merge automatically, and what stops it? Safety limits matter more when the matching logic isn't yours
How often does it look for new duplicates? It tells you how quickly new records are caught

For a comparison of chat assistants, automations and scripts, see our post on how AI tools compare for CRM deduplication. If your security team is involved, our post on what security teams ask about a deduplication tool covers their side.

How Dedupely Intelligence uses AI to find complex duplicates

Dedupely Intelligence is AI-powered matching: AI designs the matching logic, which runs as deterministic code on Dedupely's infrastructure. Your CRM data is never sent to, processed by or trained on external AI models.

Rejected matches stay rejected and don't train it. Scans run automatically, and for safety, you review and merge its matches yourself.

See what AI-powered matching finds in your CRM

Ask where the AI runs

"AI-powered" tells you AI is involved somewhere. Where it runs tells you how the tool behaves: whether results repeat, where your data goes and whether you can see why records matched.

Whichever design you choose, the decisions that matter most stay with you. Your team decides what a duplicate is, what the merged record keeps and which matches get rejected.

Frequently asked questions

How does AI find duplicate records?

In one of three ways. A language model judges records at each run, a trained model scores each pair, or AI designs logic that runs as ordinary code. Each design differs in consistency, privacy and how easy it is to explain a match.

Is AI duplicate matching accurate?

It depends on the design and your data. Logic that runs as fixed code gives the same result every time, so you can test it. Run-time models can answer differently between runs, so whatever the tool, test it on records you already know before merging.

Does AI deduplication send my CRM data to an AI model?

Some tools do and some don't. If a model judges your records at run time, your data goes wherever that model runs. If AI only designed the logic, data can stay where the code runs, so ask the vendor directly and get the answer in writing.

Can AI decide which duplicates to merge?

AI can find likely duplicates, but your team should decide what counts as one and what the merged record keeps. Those are business rules about owners, stages and key fields. Review uncertain matches, reject false ones and preview merges before anything changes in your CRM.

Should an AI deduplication tool learn from my rejections?

It depends on what you want. A model that learns from feedback can change its results over time, which some teams like and others find hard to test. What matters most is that a rejected match stays rejected and that you know whether your decisions change the model.

Portrait of Gaby Salinas

Behind the article

Gaby Salinas

Marketing Assistant at Dedupely

Gaby Salinas came to CRM data from marketing and advertising, so she looks at duplicates from two sides: the match and merge logic behind them, and what they do to the people using the CRM, like the rep who calls a customer twice, the admin who gets blamed and the cleanup nobody wants to start.

As Marketing Assistant at Dedupely, she follows how RevOps teams, end users, consultants and technical teams actually work with their data: the records they stop trusting, the workarounds they build and the moment they finally decide to clean up. What she finds shapes most of what Dedupely publishes.

Connect with Gaby on LinkedIn

Contact us

We’d be happy to help you get this set up.

Write us a message

We probably know the answer to your question already

Email copied to your clipboard!

Book a Zoom

Whether you’re getting started or getting intense.
Get in touch!

Get started for free

20 free trial merges and as much free support as you need to get your duplicates under control.

"Our customers want to achieve great results fast. Everybody at HubSpot already knows what Dedupely can do..."
Scott B

Scott B

VP Platform Ecosystem, HubSpot

Running CRM cleanup for clients? See the Partner Program

Skip to content