Being cautious and following the few simple rules detailed below will help a lot in ensuring data consistency within the database and in avoiding common errors during database update or data exports.
Most import workflows should contain the following activities: Load file , Reconciliation , Segmentation , Deduplication , Update data .
Using import templates makes it very convenient to prepare similar imports and ensure data consistency within the database.
In many projects, imports are built without Deduplication activity because the files used in the project do not have duplicates. Duplicates sometimes appear from importing different files. De-duplication is then difficult. Therefore a deduplication step is a good precaution in all import workflows.
Do not rest on assumption that the incoming data is consistent and correct, or that the IT department or Adobe Campaign supervisor will take care of it. During the project, keep the data cleansing in mind. Deduplicate, reconcile, and maintain consistency when you import data.
An example of a generic workflow template designed for importing data is available in the Example: Import workflow template section.
Note:
You can also use import templates. They are workflow templates defined by an administrator that, once activated, only offer the possibility to specify the file containing the data to import.
The most efficient format for imports is flat files. Flat files can be imported in bulk mode at the database level.
lastname;firstname;birthdate;email;crmID Smith;Hayden;23/05/1989;hayden.smith@example.com;124365 Mars;Daniel;17/11/1987;dannymars@example.com;123545 Smith;Clara;08/02/1989;hayden.smith@example.com;124567 Durance;Allison;15/12/1978;allison.durance@example.com;120987
Use zipped files for imports and exports when possible. GZIP is supported by default. You can add pre-processing when importing files or post-processing when extracting data, respectively in the Load file and Extract file workflow activities.
Regular imports must be done in delta mode. It means that only modified or new data is sent to Adobe Campaign, instead of the whole table every time.
-
Ensure that the imported data is always "normalized" (email, phone number, direct mail address) and that this normalization is reliable and will not change over the years. If this is not the case, some duplicates are likely to appear in the database, and as Adobe Campaign does not provide tools to do "fuzzy" matching, it will be very difficult to manage and remove them.
Using an import template is a best practice if you need to regularly import files with the same structure.
This example shows how to pre-set a workflow that can be reused for importing profiles coming from a CRM in the Adobe Campaign database.
-
-
-
Define the expected structure by uploading a sample file. The sample file should contain only a few lines but all the columns necessary for the import. Check and edit the file format to make sure that the type of each column is set correctly: text, date, integer, etc. For example:
lastname;firstname;birthdate;email;crmID Smith;Hayden;23/05/1989;hayden.smith@mailtest.com;123456
-
In the File to load section, select Upload a new file from the local machine and leave the field blank. Each time a new workflow is created from this template, you can specify here the file you want, as long at it corresponds to the defined structure.
You can use any of the options but you have to modify the template accordingly. For example, if you select Use the file specified in the inbound transition , you can add a File Transfer activity before to retrieve the file to import from a FTP/SFTP server.
If you want users to be able to download a file containing errors that occurred during an import, check the Keep the rejects in a file option and specify the File name .
-
-
Configure the Reconciliation activity. The purpose of this activity in this context is to identify the incoming data.
-
In the Relations tab, select Create element and define a link between the imported data and the recipients targeting dimension (see Targeting dimensions and resources). In this example, the CRM ID custom field is used to create the join condition. Use the field or combination of fields you need as long it allows to identify unique records.
-
-
Configure the Segmentation activity to retrieve reconciled recipients in one transition and recipients that could not be reconciled but who have enough data in a second transition.
The transition with reconciled recipients can then be used to update the database. The transition with unknown recipients can then be used to create new recipient entries in the database if a minimum set of information is available in the file.
Recipients that cannot be reconciled and do not have enough data are selected in a complement outbound transition and can be exported in a separate file or simply ignored.
-
Configure the Update data activity located after the first outbound transition of the Segmentation activity configured previously.
-
In the Fields to update tab, indicate the fields from the Profiles dimension to update with the value of the corresponding column from the file. If the names of the file columns are identical or almost identical to the names of the recipients dimension fields, you can use the magic wand button to automatically match the different fields.
Note:
If you plan on sending direct mails to these profiles, make sure to include a postal address as this information is essential to the direct mail provider. Also make sure that the Address specified box in your profiles' information is checked. To update this option from a workflow, simply add an element to the fields to update, and specify 1 as Source and select the [postalAddress/@addrDefined] field as Destination . For more on direct mail and the use of the Address specified option, see this document.
-
-
-
In the Fields to update tab, indicate the fields from the Profiles dimension to update with the value of the corresponding column from the file. If the names of the file columns are identical or almost identical to the names of the recipients dimension fields, you can use the magic wand button to automatically match the different fields.
Note:
If you plan on sending direct mails to these profiles, make sure to include a postal address as this information is essential to the direct mail provider. Also make sure that the Address specified box in your profiles' information is checked. To update this option from a workflow, simply add an element to the fields to update, and specify 1 as Source and select the [postalAddress/@addrDefined] field as Destination . For more on direct mail and the use of the Address specified option, see this document.
-
After the third transition of the Segmentation activity, add a Extract file activity and a File transfer activity if you want to keep track of data not inserted in the database. Configure those activities to export the column you need and to transfer the file on a FTP or SFTP server where you can retrieve it.
The template can now be used and is available for every new workflow. All is needed is then to specify the file containing the data to import in the Load file activity.
