Why not compare an n8n run against the previous run?

Because one bad day then becomes tomorrow's baseline. Comparing against yesterday means a gradual decline never trips a threshold, since each run is only slightly worse than the one before it. A median across several runs resists a single bad value in a way that a comparison against the last one cannot.

How do you catch a run with the right number of rows and the wrong rows in it?

Name the values that must be present rather than counting. A bank statement has the same regulars every month, so if rent is missing something was cut regardless of the total. No count answers this, because a pull can drop one row, gain two others and land on exactly the normal number.

What can output monitoring not catch?

A change that moves every row at once. If a source changes and all values shift together, the count is right, the expected names are present, and the workflow's own history agrees with the wrong answer. Nothing that compares a run against its own past can see it; catching it needs a known-good value from outside the run, which in practice means a person checking a handful of records.

How to catch an n8n workflow that runs but produces the wrong output

Q: Can a rolling baseline learn an outage as normal?

Yes, and this is the main argument against learning the baseline at all. If a workflow returns nothing for two weeks, a rolling median of recent runs becomes zero, the outage stops looking abnormal, and the recovery is what pages you. The fix is to let history propose a number and a person approve it, because an approved number cannot drift.

Every check on this page compares a completed, successful run against what that workflow normally produces. None of them look at whether it errored, because none of these runs error. This is the layer above did it run and did anything come out.

Each design below exists because somebody who runs this for a living said the previous version was not good enough. The order is the order they arrived in.

How do you alert when a workflow returns fewer rows than usual?

Compare each run's item count against a median of that workflow's own recent runs, not against a fixed floor. Take the count from the last node that executed rather than summing every node, or a three-node chain passing a hundred rows reports three hundred and the figure moves every time somebody adds a node.

A fixed floor catches zero and nothing else. The case that costs money is the pull that returns sixty per cent of normal, which passes every non-empty assertion anyone has ever written. It looks like a quiet morning.

Give the baseline a calendar where the data has one. A client whose Monday is legitimately ten times its Tuesday will either alert every Monday or hide a collapsed Monday inside the week's average. Comparing Monday against other Mondays fixes both, and needs enough same-weekday history before it is allowed an opinion.

Why not compare against the previous run?

Because one bad day becomes tomorrow's baseline. If each run is measured against the last one, a gradual decline never trips anything, since no single step is large enough to notice.

A median across several runs resists one bad value in a way that a comparison against the most recent cannot. It is also why an average is the wrong statistic here: one enormous day drags the mean somewhere no real run has ever been.

Can a rolling baseline learn an outage as normal?

Yes, and it is the strongest argument against learning the baseline at all. If a workflow returns nothing for two weeks, the median of recent runs becomes zero, the outage stops looking abnormal, and the recovery is the thing that pages you.

The answer is not a cleverer statistic. It is to let history propose a number and a person approve it. An approved number does not drift, cannot learn an outage, and cannot quietly re-centre on a bad value.

The cost is real and worth stating: an approved number does not follow a legitimate change either. Double your client's volume and the check complains until somebody updates it. That is the correct trade for anything that pays the bills, and it is a trade rather than a free win.

runs:      52  49  51  0  0  0  0  0
median of the last eight ...... 0     <-- an outage became normal
the number you approved ....... 50    <-- did not move

How do you catch the right number of the wrong rows?

Name the values that must be present, and check for those instead of counting. A pull can drop one row, pick up two others and land on exactly the normal total, at which point every count in the system agrees with itself and the data is wrong.

The clearest statement of this came from an agency owner running a bank-statement pull, in a public thread on r/n8n:

a statement has the same regulars every month, rent, salaries, two or three subscriptions. if they are not in the pull then something got cut no matter what the total says. percentages never caught that, missing names did.

Values that must appear in every run

count normal · a name missing

A list of values, and the column to look for them in. Matching is case-insensitive substring, because real descriptions are dirty and RENT PAYMENT 4421 has to satisfy Rent.

Naming the column matters. The word Rent appearing in a memo field is not the rent line arriving, and a check that searches every column will happily reassure you about the wrong thing.

A list that has gone stale

renamed, not lost

The same operator came back a day later with the failure that kills this idea: one of mine renamed itself and the check cried every morning until i stopped reading it.

A muted check is worse than no check, because it hides the real ones too. So a value absent from this run and from every earlier run readable is reported as renamed, with one line telling you to update the list and saying plainly that it will repeat until you do. That is an instruction. A value that was there last week and is gone today is the alarm.

Values that only turn up occasionally

not every run

A monthly line in an hourly pull is absent from almost every run. Put it in the every-run list and it alerts on nearly all of them.

So there are two lists. Things that must be in every run, and things that must turn up at least once inside a window — rent monthly, insurance quarterly. The second list is what catches the quiet things, at the cost of taking a month to say anything.

The shape of what each node emits

success · fields gone

Below the totals, each node's output has a shape: how many items, which keys, and what type each value is. An upstream API dropping a field, or returning a number where it used to return text, changes the shape without changing the count.

Type changes are only reported when every earlier run agreed, because a field that has always been mixed is not evidence of anything, and a check that alerts on ordinary variation gets muted like any other.

What can none of this catch?

A change that moves every row at once. If a source changes and all the values shift together, the count is right, the expected names are all present, and the workflow's own history agrees with the wrong answer.

An operator who scrapes prices put it exactly: a site-wide change moves every row together, so your own history looks fine. A second operator, in a different thread, arrived at the same place from the other direction: a count can look correct while every row is the wrong one.

Nothing that compares a run against its own past can see this, and that includes everything above. Catching it needs a known-good value from outside the run. In practice that means a person: the price scraper keeps about five URLs whose values he checks by hand once a month.

I do not have an automated answer to this and I am not going to pretend otherwise. The expected value has to come from somewhere the change cannot reach, and a number the tool can compute drifts with the same change that broke the data.

I do fixed-fee reliability audits for n8n and Make automations — one week, a written report naming which of your workflows fail silently today and what breaks first.

There is a full sample, run against my own production stack and published unedited, including the five silent failures it found there.

Read the sample audit

Scope

Every check described here is implemented and open source: github.com/moneywithjjcom-del/ranfine-. It reads execution history through the public API with a read-only key, adds no nodes to any workflow, and runs nothing inside the instance.

Comparisons are made only between runs of the same workflow version, so editing a workflow resets its baseline rather than alerting on itself. That has an honest cost: an actively edited workflow rarely accumulates enough same-version history for these checks to have an opinion, and a baseline built across an edit is worse than no baseline.

Pruned execution data is treated as unknown throughout, never as zero. n8n ageing out history is not a workflow producing nothing, and a check that confuses the two alerts on everything the moment the retention window rolls over.

Running these checks across instances you do not own is a different problem, covered in monitoring client n8n instances without touching them.

The quotations are from public threads on r/n8n and are linked above. The operators are described rather than named, because they were talking to each other rather than writing marketing copy for me.