Duplicate Rules and Matching Rules in Salesforce

March 20, 2024: This article has been updated to include Data Cloud. 

If your organization uses Salesforce, chances are that you’ve grappled with duplicate records at some point. With data coming from many sources such as direct user entry, imports using the Salesforce Data Wizard or third-party integrations, it’s pretty difficult to maintain good data hygiene. That being said, there is an out-of-the-box functionality that allows you to de-duplicate directly in Salesforce, the Salesforce Duplicate Management feature.

The duplicate management feature gives you a way to define how your organization identifies duplicates and what Salesforce should do when a duplicate is detected. The functionality relies on two main components:

Salesforce Matching Rules: Matching rules are used to determine whether the record that is being created or edited by the user is, in fact, a duplicate. For example, if two contacts have the same email address they will be classified as a duplicate if the matching rule for email addresses is‘exact’ or ‘fuzzy’.

Salesforce Duplicate Rules: Duplicate rules are used to define the action that Salesforce should take when the user is editing or attempting to create a duplicate record. For example, a duplicate rule can be customized to block users from saving records that meet Salesforce standard matching rules criteria, or simply alert the user that they are potentially creating a duplicate record.

In an ideal world, matching and duplicate rules should be deployed upon implementing Salesforce. If you’re like most users, deploying rules in a mature environment, consider running a Salesforce duplicate job to get a list of records that meet the criteria outlined in the rules.

Salesforce Duplicate Jobs are an integral part of the Salesforce Duplicate Management system, they’re designed to scan your existing records to find duplicates and are immensely useful in large datasets where manual detection is nearly impossible. Executing a duplicate job is straightforward. You first select the object (like Contacts or Accounts) and the matching rule you want to use. Then, Salesforce will scan the selected object’s records using the defined rule and compile a list of potential duplicates. The results of a duplicate job are visible in the Duplicate Record Sets page, where each set contains the potential duplicate records. By having this overview, you can review the suggested duplicates and decide whether to merge or dismiss them, providing greater control over your data’s integrity.

Note, Duplicate Jobs are only available in Unlimited and Performance Editions of Salesforce and can only scan a maximum of 50 million records. If your dataset exceeds this limit, Salesforce Deduplication will only scan the most recently updated records. Also, the duplicate job doesn’t stop new duplicates from being created; it merely helps identify existing duplicates.

While Salesforce Duplicate Management provides a good starting point, it may fall short of what you need to eliminate duplicates completely.  Here are some limitations that you should be aware of:

1. Only five duplicate rules can be deployed on each object (leads, contacts and accounts). The max rule limitation makes it nearly impossible to have a rule that covers all duplicate scenarios.
2. Duplicate prevention rules may result in data loss when information is passed from third-party integrations. For example, if you pass records into Salesforce from a marketing automation system, they can be blocked from being created and the additional intelligence/field data may also be lost.
3. Users will still need to merge Salesforce accounts, leads, and contacts manually, and the usual out-of-the-box limit for merging up to three duplicates at a time still applies.
4. You can bypass Salesforce sharing rules when activating duplicate rules.  In this event the rule may trigger a duplicate alert, but the user will not be able to see and/or edit the duplicate if they do not have access to the record.
5. Given that duplicate jobs are only available in Performance and Unlimited Editions of Salesforce, organizations using other editions will need to look at alternatives for data created prior to deploying duplicate rules and matching rules.

A critical factor that has altered the landscape since we first released this blog is the introduction of generative AI technologies, such as Salesforce Einstein and ChatGPT. Einstein needs data. While your Salesforce org has plenty of good information about your customers, it is almost certain that your organization has other data silos that can be used by Einstein to generate a more complete picture.  Hence, Salesforce Data Cloud has come to prominence. 

Let’s start off by pointing out that Data Cloud is not a deduplication tool.  It will not merge or remove duplicates.  Data Cloud will ingest data from Salesforce and external sources. Once this data is available, Data Cloud will identify information that is related to a specific individual and then select the “best value” for a unified field and the “best profile” based on standard matching criteria.  

Here is an overview of Identity Resolution Rulesets: 

• Identity Resolution Match Rules “unify” profiles to eliminate duplicates.  They are limited to specific objects such as Accounts and Individuals and support fuzzy and exact matching.

• Identity Resolution Reconciliation Rule selects the best value to save to a unified field during the identity resolution process.

Given the limitations of the out-of-to-box Salesforce duplicate management feature and the time involved in merging the resulting duplicates, you may want to consider a third-party Salesforce duplicate management tool.

Using a third-party tool, like DataGroomr, that uses Machine Learning algorithms to automate duplicate detection allows you to sidestep creating rules and filters and gives you the ability to merge duplicate Salesforce records efficiently. In addition, if you’re looking for replacement of Salesforce duplicate jobs, DataGroomr comes with the export duplicates feature that can populate Duplicate Record Sets.

Want to try it? Click the ‘Free Trial‘ button at the top right-hand side of this post.