Lesson 10 / 25

Loops and Batches

Process many items in batches to respect API limits.

Do not hammer the API

By default a node processes all items quickly, which may hit a service's rate limit. The Loop Over Items (Split in Batches) node feeds items in groups, for example 10 at a time; add a Wait node inside the loop to pause between batches. Most nodes also have batching and retry settings under their options.

Batching arithmetic

This chunk function ran in Node.js and shows what a batch size of 2 does to 5 items: three groups, the last one smaller.

function chunk(arr, n) {
  const r = [];
  for (let i = 0; i < arr.length; i += n) r.push(arr.slice(i, i + n));
  return r;
}
console.log(JSON.stringify(chunk([1, 2, 3, 4, 5], 2)));

Output:

[[1,2],[3,4],[5]]

Quick check: Why process items in batches with a Wait node?

  • To stay within an API's rate limit
  • To make data larger
  • Because n8n forbids loops
  • To hide errors
Answer

To stay within an API's rate limit — Spacing out calls prevents 429 errors from services with request limits.