Lesson 11 / 25

Filter, Sort, Aggregate and Remove Duplicates

Use built-in data nodes before reaching for the Code node.

Prefer a built-in node when one exists

n8n has ready-made nodes for common data jobs: Filter keeps items matching a condition, Sort orders them, Limit keeps the first N, Remove Duplicates drops repeats by a key, Aggregate combines many items into one list, and Summarize computes counts, sums and averages. They are easier to read and maintain than equivalent Code, and teammates who do not code can edit them.

A typical daily report chain

Each line is a node. The AI step receives one compact summary rather than hundreds of raw rows, which also saves tokens.

HTTP Request        fetch yesterday's orders (200 items)
 -> Filter           status is "paid"
 -> Remove Duplicates  by order id
 -> Summarize        count, sum(total) grouped by region
 -> Aggregate        all groups into one list
 -> Basic LLM Chain  "Write a 3-line sales summary from this data"

Summarise before the AI step

Sending hundreds of raw rows to a model is slow, costly and noisy. Reduce the data with Filter and Summarize first, then give the model only what it needs to write or decide.

Quick check: Which node combines many items into a single list item?

  • Wait
  • Error Trigger
  • Webhook
  • Aggregate
Answer

Aggregate — Aggregate merges the incoming items so the next node, such as an LLM, sees them together.