Deduplicate

Overview

The Deduplicate node removes duplicate items from an incoming list, emitting only unique entries based on a specified key path. It preserves the first occurrence of each unique value by default, discarding subsequent duplicates. Use it after merge or aggregation steps where repeated entries may appear — for example, deduplicating battle results by model ID before scoring. Items that cannot be resolved against the key (missing key, null value) are routed to the unresolved output port rather than silently dropped.

Configuration

Field	Type	Required	Description
key	string	Yes	Dot-notation path to the field used to determine uniqueness (e.g. id, model.id, result.runId). Two items are considered duplicates when this field resolves to the same value.
strategy	enum	No	Which occurrence to keep when duplicates are found. first (default) keeps the earliest item; last keeps the latest item in the list.
caseSensitive	boolean	No	When true (default), string key comparisons are case-sensitive. Set to false to treat ModelA and modela as the same key.
emitUnresolved	boolean	No	When true (default), items where the key path cannot be resolved are forwarded to the unresolved output port. When false, they are silently discarded.

Inputs

Port	Type	Description
input	array of objects	The list of items to deduplicate. Each element should be an object; primitive arrays are not supported.

Outputs

Port	Type	Description
output	array of objects	The deduplicated list, containing only the kept occurrence of each unique key value, in original order.
unresolved	array of objects	Items for which the configured key path could not be resolved (missing field or null value). Only emitted when emitUnresolved is true.

Example

{
  "nodeType": "deduplicate",
  "config": {
    "key": "model.id",
    "strategy": "first",
    "caseSensitive": false,
    "emitUnresolved": true
  }
}