Deduplicate Array

This processor deduplicates one or more arrays of any array type.

The following table describes the configuration options:

Configuration Description

Configuration	Description
Inputs	Select the arrays that you want to deduplicate. They can be of type String Array, Number Array or Date Array. Each array is deduplicated independently (i.e. duplicates can exist across different arrays).
Outputs	Resulting arrays after deduplicating inputs, one per input. `[Input Attribute].Deduplicated`: the resulting array after removing duplicates `[Input Attribute].DataDeduplicated`: indicates if the deduplication has occurred or not.

Inputs

Select the arrays that you want to deduplicate. They can be of type String Array, Number Array or Date Array. Each array is deduplicated independently (i.e. duplicates can exist across different arrays).

Outputs

Resulting arrays after deduplicating inputs, one per input.

[Input Attribute].Deduplicated: the resulting array after removing duplicates
[Input Attribute].DataDeduplicated: indicates if the deduplication has occurred or not.

Output Filters

Configuration	Description
Deduplicated	Any records that has an element removed.
Unique	Any records where all the elements were already unique and no change was made.

Example

Single Array

Array	String	Separator
(John)(Smith)	(John)(Smith)	Unique
(Bank)(Automobile)(Bank)	(Bank)(Automobile)	Deduplicated
(Zipcode)(<Null>)(<Null>)(<Null>)	(Zipcode)(<Null>)	Deduplicated
(London)(Blank)(Null)(Blank)	(London)(<Blank>)(<Null>)	Deduplicated
<Null>	(<Null>)	Unique

Two Array

Input Value A	Input Value B	Output Value A	Output Value B	Output Filter
(A)(B)	(B)(C)	(A)(B)	(B)(C)	Unique
(C)(A)(B)(A)(A)	(D)(D)(D)	(C)(A)(B)	(D)	Deduplicated