Combine and Merge Duplicate Records

More information

The Combine/Merge Duplicate records feature ("Combine/Merge Dups") allows you identify and merge duplicate individual or organization information into one master record.

Usage

The Combine/Merge Dups functionality provides a way to:

  • Search for records via query or a find
  • Select the duplicate records
  • Select master record
  • Select and merge specific data into the master record
  • Hide or delete the other records

Note: The process for merging duplicate Individual and Organization records is the SAME.

To Search for Duplicate Records:

  1. In the Individuals group item, select the Combine/Merge Dups group item link.
  2. The Combine/Merge Duplicate Records window will display.
  3. Select Individual or Organization from the Combine Information For drop-down list.

  1. Choose either Query Individuals or Find Individuals.

    2. Note: By default, individuals are listed in alphabetical order by the first column in the list, which is the individuals' last names. If you would like the sort order to be changed, contact your administrator. Administrators, see Technical Information for specific instructions.

  2. On the Find or Query page, enter search criteria to find the potential duplicate individual records.
  3. After you enter the search criteria, click Go. A list of duplicate or similar records displays.

To Select the Duplicate Records:

  1. On the Combine/Merge Duplicate Records search results page, select the check boxes next to the potential duplicate records.
  2. Click Continue/Refresh.
  3. Scroll down the page to see your next options.

To Select the Master Record:

  1. After you select the duplicates and click Continue/Refresh, the records you selected display side-by-side. Select the Keep radio button next to the record you want to make your master record. (After the record information is merged, the other records will be hidden.)
  2. Click Continue/Refresh.
  3. Scroll down the page to see your next options.

To Select and Merge Specific Data Into the Master Record:

  1. Under the records that you selected to delete or hide are the same records displayed in a different format. This time, the records are shown with check boxes for the different netFORUM categories.
  2. Select the appropriate check boxes for the data that you want to merge into the master record (ALL check boxes will be selected as the default).

    3. Note: The data selected is not checked for duplicates. If you select address as data to be merged, then the system will merge the address records from the hidden/deleted record to the kept record. This may result in multiple address records with the same information.

  3. To merge selected data into the master record, click Continue and Combine/Merge Records (You will see a progress message as the record is updated).

Note: Do not close the browser before the processing is done or the merge will be lost.

  1. When the process is complete, you will see another message that says all file groups have been processed. Click Close Window.

  1. To verify that the records were merged, search for the record (Find, List, or Query). When you view the List of Individuals, you will see that there is just one record now, the master record.

Additional Information

  • Your system administrator can adjust the functionality of the combine/merge dups process by going to the Admin module, md_dedup_table, and specifying the tables and customer key columns to be used for the combine/merge duplicates process.
  • There is a bug for the combine and merge function where you can't merge Advertising information. See bug # 4420 for more information.
  • On the Log tab of the customer who is "kept" after a merge, you can view a list of any merged customers who were hidden on the merged individuals (or organizations) child form. Customers who were deleted will not appear on that child form.

FAQ

Unmerge Merged Records

The NetForum Combine/Merge feature does not have a way to un-merge merged records. It would be very difficult to make this happen because when records are merged, some of the records from the duplicate customer are deleted outright, some are hidden, and others are transferred to the "surviving" customer. To put all of this back together again pre-merge would be difficult, and the deleted records are unrecoverable.

One partial solution is this: assuming you chose the "hide" option for the "duplicate" customer (rather than "delete"), this customer is now hidden. You could unhide that customer which will essentially bring that customer back. Any related records from that customer that were deleted or transferred, however, would not be recoverable. For example, you might need to re-add committee assignments, memberships, etc. The main value of this approach is that the "duplicate" customer would still keep their original Customer ID if that's important to you.

Identifying Duplicates

The Combine/Merge tool does not identify duplicates, it only merges records that you have already determined to be a duplicate. The Individual Duplicate Check Report can be used to help identify duplicates. This report provides a list of potential duplicate individuals, based on several different duplicate-matching algorithms that you can specify in a run-time parameter.

Technical Information

Q. I get this message - DELETE statement conflicted with COLUMN REFERENCE followed by a database table name and other information. Example:

A. The Combine Merge Process attempts to delete certain records of the record you want to drop. In the course of doing so, it is possible that certain dependent records could become "orphaned". For example, if you are attempting to merge two organizations, and one of the organizations has relationships with individuals, then it is not possible to delete that organization. The enforcement mechanism is done with database foreign key constraints, which enforce referential integrity in a database.

In order to resolve this issue, go to the record you want to delete and manually move the dependent records.

In the example in the above screen shot, the record being deleted is a Customer record. But based on the column adr_cst_key_owner, the organization has an "owned" or "linked" address .

Q. How does this feature know which data to use?

A. See Dedup Table. Dedup Tables list which tables should be updated when merging.

Q. How do I adjust the column order, and therefore sort order in the Find Individuals page of the Combine/Merge Duplicate Records process?

A. First, you must have administrative rights in iWeb. With administrative rights, go to Toolkit > Tables List and edit the list table definition for co_individual. Go to the list columns that will appear in list results child form. Select or add any of the columns containing the following types of information and move the selected column to the top of the list:

  • any name information
  • any information on individual type
  • any code information
  • any ID

The Combine/Merge Duplicate Records page sorts by the contents of the column at the top of the list.

Caution! Changing the setting for co_individual changes the column order in any pages that list individuals. The sort order is only affected by this change if there is no other way to set sort order on the page.

FAQ

Organization Demographics

Q. What happens if two organizations are merged and each has custom demographics?

A. The surviving organization keeps its demographics and the one that gets hidden or deleted will lose those demographics. It's not possible to merge the two sets of demographics as they might be in conflict with each other.