Salesforce 101: Salesforce Duplicate Management

Why Is Duplicate Data a Problem?

Everyone agrees that duplicate data is a problem. I’ve never spoken to a client that simply agrees to live with duplicate data. No one wants it — and yet it is a massive problem. If no one wants it, and it’s a massive problem, why does it persist?

One explanation lies in the choices businesses make and what is prioritized as “important.” (I have ninety-nine problems, and dirty data might be one of them, but excuse me, I have sales targets to hit).

But kicking the proverbial data can down the road will only hurt company performance. Duplicate data can cripple sales, marketing, and customer success strategies. I’ve outlined that in a previous post.

At the end of the day, “data management” simply takes a back seat to other company initiatives because we make that choice. We choose to de-prioritize it. We think it isn’t important or we think it is too difficult to solve, and we don’t fully appreciate the cost of not solving it.

But the reality is that all other major company initiatives will have better outcomes if data hygiene is taken seriously.

If you use Salesforce as your primary CRM, the good news is you can get started (today!) on a path to prioritizing the management of your customer data.

In this article, we will focus primarily on resolving duplicate issues, but stay tuned for other helpful data integrity and hygiene content in the near future.

How can out-of-the-box Salesforce Duplicate Management tools solve my duplicate records?

Salesforce comes with important duplicate management features that work in tandem to help manage duplicates. Let’s call these the “core features” of the “Salesforce Duplicate Management System.”

Yes, there are a lot of other features and nuances to cover within this system, but I’m going to keep things relatively simple for now.

Let’s meet the major players in the Salesforce Duplicate Management arena:

• Duplicate Rules – contain the actions taken when a duplicate is found
• Matching Rules – contain the duplicate definitions
• Duplicate Record Items – the records Salesforce has identified as potential duplicates using Duplicate Rules and Matching Rules
• Duplicate Record Sets – contain the potential duplicate records. This is where you can merge duplicates in Salesforce. A Record Set is literally a “set” or “container” of Duplicate Record Items.
• Potential Duplicates Page Component – alerts users to a potential duplicate while they are on a record page.
• Reports & Dashboards – helps easily identify duplicate records org-wide and take action.

Duplicate Rules & Matching Rules in Salesforce

A Duplicate Rule uses Matching Rules to find potential duplicates. You can think of it this way — the Duplicate Rule is the bow from which the arrow (Matching Rule) is shot. The target that the arrow strikes is the duplicate record.

This duplicate record can be a Lead, Account, or Contact. As Matisyahu said in Time of Your Song, “I’m the arrow, you’re the bow, shoot me forth, and I will go (find a potential duplicate).”

Out of the box, Salesforce takes this duplicate rule design work off your plate by presenting you with pre-fabricated bows and arrows using Duplicate Rules paired with the Matching Rules below.

Alert! These out-of-the-box rules may look different based on what edition of Salesforce you have. If you don’t see all of these on your list, and you need help configuring Duplicate Rules, reach out to us.

Although Salesforce has very sophisticated Matching Rules containing esoteric “Match Keys”, “Match Equations,” and “Match Criteria” (all of which can be perused here), I’ve provided my own short-hand descriptions below.

1. Standard Contact Duplicate Rule
   • Contacts that look like other Contacts
2. Standard Lead Duplicate Rule
   • Leads that look like other Leads
3. Standard Rule for Leads with Duplicate Contacts
   • Leads that look like Contacts
4. Standard Rule for Contacts with Duplicate Leads
   • Contacts that look like Leads
5. Standard Rule for Duplicate Accounts
   • Accounts that look like other Accounts
6. Standard Rule for Person Accounts
   • Person Accounts that look like other Person Accounts (only useful if you are, in fact, using Person Accounts in your Salesforce Org).

Duplicate Record Items & Duplicate Record Sets

Now, this is the fun part! Continuing the metaphor above, when the arrow strikes a target, how do we know and where do we find it in Salesforce? Well, in Salesforce terminology, a “strike” is called a Duplicate Record Item. And you can build reports that you can add to a dashboard to easily monitor Duplicate Record Items and manage them.

Let’s dive a little deeper into these features…

The first step to building a duplicate report — whether for Leads, Contacts, or Accounts — is understanding that when you have a strike (match) on a target (potential duplicate), a Duplicate Record Item gets created. This in turn creates a Duplicate Record Set. A Duplicate Record Set is simply the “set” of potential duplicates that were found or matched. For example, three Contacts that have the same email, or two Accounts with the same Account Name, and so on.

This can all be restated as follows (repeat after me): Duplicate Rules create Duplicate Record Items which create Duplicate Record Sets.

If this seems overwhelming, don’t worry — this is all happening automatically, behind the scenes for you. All you have to do is create some reports to visualize it.

Create your Duplicate Reports

The astonishing thing about Duplicate Reports is that they are not set up out of the box in Salesforce. You have to create a Custom Report Type for each Duplicate Report you need.

For most people, you will need Leads, Contacts, and Accounts Duplicate Reports.

Step 1: Navigate to Setup

Step 2: Select Report Types

Step 3: Select New Custom Report Type

Step 4: Choose your Primary Object (this should be either the Lead, Contact, or Account object)

Step 5: Choose your Secondary Object (this will always be the Duplicate Record Item object)

Step 6: Save

Repeat this process until you have three custom report types — one for each “Primary Object” – Leads, Contacts, and Accounts.

Step 7: Go to Reports, find the duplicate report you just built, and select “Start Report”.

Step 8: Group the Report by Duplicate Record Set Name, and add a few more fields for ease of use. Make sure to add a filter for “Duplicate Record Sets > 1”.

If you are a report wizard, you can get creative here and add different groupings or filters as you see fit. For example, I sometimes group this report by Create Date first so I can monitor duplicates on a daily, weekly, or monthly basis.

Step 9: Save & Run the report to view your finished product.

Merge Duplicate Records via the Duplicate Record Set

Now that you’ve successfully identified potential duplicates (contained within your Duplicate Record Sets), you can now use those Duplicate Record Sets to merge the potential duplicates within.

Step 1: Open a Duplicate Record Set

Step 2: Select the “Compare and Merge” button at the top right

Step 3: Proceed through the wizard to merge the records

Add your Duplicate Reports to an Audit Dashboard

Create an “Audit Dashboard” (I’ve outlined other items you should add to your audit dashboard in a previous post), and add all of your duplicate reports to it. This will help you as you clean up existing duplicates and monitor duplicate creation moving forward.

Add the “Potential Duplicates” Component to Page Layouts

In conjunction with the features previously described, you can also add a “Potential Duplicates” component to your Lead, Contact, and Account page layouts through the Lightning Page builder.

Step 1: Select Setup, and then Edit Page

Step 2: Find the Potential Duplicates component

Step 3: Drag & Drop onto a place you’d prefer on the page

Step 4: Save (at the top right) & Activate the new page edit

Step 5: See it in action — when a potential duplicate is detected, the page component will alert you. View the duplicates to kick off the merge wizard.

What about custom objects?

The great benefit of Duplicate Rules and Matching Rules is that they are available for you to use for standard and custom objects.

My examples thus far have only included Leads, Contacts, and Accounts. But you can also create custom Duplicate Rules and custom Matching Rules for other standard objects such as Opportunities, Quotes, Contracts, and Cases (just to name a few).

You can also apply Duplicate Rules to custom objects that you create.

For example, I have a client that sells software. They use two custom objects: a Project custom object that contains Project Tasks. These objects help them manage all the tasks related to onboarding and implementation after an Opportunity is Won. It is basically a custom-built task manager for them in Salesforce.

If they wanted to create a Duplicate Rule with Matching Rules that prevented duplicate Projects and Project Tasks from being created, they could!

From this perspective, Salesforce duplicate management features, while most often applied traditionally to Leads, Contacts, and Accounts, are not limited to those only those objects.

You can create a system that monitors duplicate creation across all objects in the org, if you wanted to!

This article will not go into the finer details of how specifically to create custom Matching Rules and Duplicate Rules (there simply isn’t enough room or time here), but you can reach out to me for a deeper dive.

What about cross-object matching & duplication?

Cross-object matching comes into play when you have two records that exist in different objects but contain the same information. The classic example of this is a Lead and Contact that share the same information but they are in separate objects (Lead and Contact objects, respectively). In other words, a duplicate, but not the same type of duplicate as one that exists in the same object (for example, two Accounts that have the same Account Name).

You actually cannot merge a Lead and Contact. Yes, you can merge a Lead with another Lead, and a Contact with another Contact, but you can’t merge a “cross-object” duplicate.

To do so, you would need to convert the Lead via the standard Lead Conversion process in Salesforce, and upon conversion choose to merge the Lead with the existing Contact.

Alternatively, you could just decide to remove either the Lead or the Contact, and keep the record that remains.

How to prevent duplicate records in the first place?

Here are a few best practices to prevent your business from continually creating new duplicates. Take heed and cut off duplicate creation at its source.

1. Don’t allow duplicates to be created in the org (modifying Duplicate Rules to use the “block” instead of “allow” feature when users create new records)
2. Properly clean and dedupe data before importing or migrating it to Salesforce
3. Check existing third-party API integrations to ensure they are built to recognize Salesforce Duplicate and Matching Rules

Signs It’s Time to Look for Other Solutions

This merge process is all well and good but you might have noticed a few inefficiencies.

• It is a manual process, which can be very time-consuming and arduous if you’ve identified thousands of Duplicate Record Sets across your Leads, Contacts, and Accounts, and you want to merge them. There is no way to “mass” merge duplicates out of the box in Salesforce.
• On that note, if you’ve tried a few merges using the wizard above, you’ve probably noticed that choosing which record to “use as the principal” or “master” record, and which data from other records to keep, takes additional time and thoughtfulness, and can be quite cumbersome if you are working with thousands of duplicate records.
• You can’t automatically merge duplicates that are identified via your Duplicate Rules and Matching Rules.
• Cross-object duplication merging adds additional manual labor to the process because now you are having to potentially convert Leads in order to merge them with an existing Contact.

If you are strapped for time (who isn’t) or don’t have administrative staff devoted to this kind of cleanup work, you should either start devoting that staff (today) or review other solutions.

Salesforce Duplicate Management Tools on the AppExchange

None of the tools below is a panacea to fix all of your duplicate woes. I recommend you reach out to each team that built each app to get a demo and ask questions.

Once you make a decision, make sure the development team is available to help you learn how to properly use their application. Look for a tool that also has a lot of help documentation online that you can reference so you aren’t stuck on an island.

Cloudingo, in my experience, is particularly powerful and efficient but it requires discussions with the Cloudingo team on how to properly use the application. It also comes with the highest price point, but I’d argue the price tag is worth it to eliminate duplicates, and it will put you on a starting path to an operative duplicate management system.

Free Tools

1. No Duplicates by At home
2. Duplicate Check for Salesforce by Plauti

Paid Tools

1. Cloudingo by Symphonic Source (starting at around $1,000/year)
2. Duplicate Management by Machine Learning by DataGroomr, LLC (starting at $99/year but the price quickly ramps up depending on the number of duplicate records and users)
3. Smart Duplicate Manager by Peeklogic (starting at $20/month)

Review the full list of free and paid options here on the AppExchange.

Other interesting ideas and parting thoughts

Apex custom code – since Apex is basically “god mode” in Salesforce, you can write custom code to help with duplicate management.

For example, you could trigger an automated email to a Sales Manager, Sales Rep, or data team as soon as a duplicate record is created. This would allow for quick action to be taken to merge the duplicate before it has a chance to cause chaos.

Duplicate Jobs – this feature is only available in Performance or Unlimited editions of Salesforce. It is basically a scan that you can run at any time. You select the Matching Rule and scan the org.

This is an important feature because simply activating a Duplicate Rule does not run a scan on all existing records. Duplicate Rules only run when individual records are created or edited. But with this Duplicate Jobs tool, that’s no longer a problem.

Salesforce Duplicate Management Explained!

I hope this information convinces you that data deduplication is possible with out-of-the-box Salesforce features.

I also hope this will help you prioritize the importance of clean data in your Salesforce org — especially as it pertains to managing duplicate records and eliminating duplicates once and for all!

Disclaimer! Each edition of Salesforce is different in terms of what might be “out-of-the-box.” So if you work through this article and run into roadblocks don’t fret! You can reach out to me or comment on this article. And also don’t forget to also try Salesforce help documentation when you get into a bind!

Finally, look out for future articles about Salesforce duplicate management — especially as it pertains to better explanations about how to create your very own custom Duplicate Rules and Matching Rules.