Learn to use the merge functionality within the deduplication activity

Introduction

This document describes how to use of the Merge functionality in the Deduplication activity in Adobe Campaign Classic.

 

Description

The Deduplication activity is used for removing duplicate rows from a data set. For example, the records below could be considered duplicate as they have the same email address and same mobile and/or home phone.

Date First Name Last Name Email Mobile Phone Phone
2/3/2013 Bob Tisner bob@mycompany.com 444-444-4444 888-888-8888
5/19/2013 Robert Tisner bob@mycompany.com 444-444-4444

777-777-7777

7/22/2013 Bobby Tisner bob@mycompany.com 444-444-4444 777-777-7777

In the earlier versions of Adobe Campaign, the Deduplication activity had the capability to keep an entire row as the unique record after duplicates were identified. For example, in the above use case, if the activity was configured to keep only the record with the oldest Date, the result would be:

Date

First Name

Last Name Email Mobile Phone Phone
2/3/2013 Bob Tisner bob@mycompany.com 444-444-4444 888-888-8888

The master record selected will carry forward the data without any merging of field data with other relevant data in the duplicate rows.

Complement:

Date First Name Last Name Email Mobile Phone Phone
5/19/2013 Robert Tisner bob@mycompany.com 444-444-4444 777-777-7777
7/22/2013 Bobby Tisner bob@mycompany.com

444-444-4444

777-777-7777

Starting Campaign Classic v6.1, an improved deduplication functionality known as Merge was introduced. With Merge, the user can configure a set of rules for the deduplication to define a group of fields to merge into a single resulting data record. For example, with a set of duplicate records, you can choose to keep the oldest phone number or most recent name.

For this capability, Advanced Parameters were added in the Deduplication activity:

dedup1

By clicking on the Advanced Parameters link, a new dialog is opened as shown below:

dedup2
  • Merge records enables new merging functionality 
  • Use serveral record merging criteria allows grouping of multiple data fields in each merging condition. 

Checking the Merge records also enables the new Merge tab. This tab allows the user to define groups of fields to merge and their associated rules.

dedup3

Use Case

The data shown below is duplicated based on the Email field. 

Data

Date First Name Last Name Email Mobile Phone Phone
2/3/2013 Bob Tisner bob@mycompany.com 444-444-4444 888-888-8888
5/19/2013 Robert Tisner bob@mycompany.com   777-777-7777
7/22/2013 Bobby   bob@mycompany.com   777-777-7777

Rules

  • Dedup on email
  • Keep the most recent name (first name and last name)
  • Keep the most recent mobile phone
  • Keep the oldest phone
  • All fields in a group must be non-null to be eligible for the final record

Configuring rules

  1. Keep the most recent name (first name and last name).

  2. Indicate the identifier of the group of fields to be merged.

    dedup4
  3. Indicate the conditions for selecting the records to be taken into account.

    dedup5
  4. Sort on date to select most recent first and the last name.

    dedup6
  5. Select the data to be merged.

    dedup7
  6. This rule is then added to the set of rules and a new element is added to the workflow schema.

    dedup8
    dedup9

Results

Eventually when the user configures all the rules in the Merge tab, the following data is received at the end of the Deduplication activity.

The result is merged from the three records as per the rules defined in the merge functionality. After comparison, it is concluded that the most recent first name, last name, and mobile phone are used, along with the original phone (Home) used to build the data. 

Original data

Date First Name Last Name Email Mobile Phone Phone
2/3/2013 Bob Tisner bob@mycompany.com 444-444-4444 888-888-8888
5/19/2013 Robert Tisner bob@mycompany.com   777-777-7777
7/22/2013 Bobby   bob@mycompany.com   777-777-7777

Result from the merge functionality

Date First Name Last Name Email Mobile Phone Phone
2/3/2013 Bobby Tisner

bob@mycompany.com

444-444-4444

888-888-8888