Data observability: catching the data incidents nobody wrote a rule for

A dashboard is wrong for three weeks before anyone notices, and by then it has fed decisions. The pipeline did not error. No quality rule failed, because no one thought to write a rule for this particular failure. This is the gap data observability fills: monitoring the data itself for anomalies nobody anticipated. This article covers the five pillars of observability, how it differs from quality rules, and how to keep it from drowning teams in alerts.

What data observability is

Data observability is the practice of continuously monitoring the health of data in production, the way application observability monitors the health of services. Instead of checking whether a server is up, it checks whether the data flowing through the platform is behaving normally: arriving on time, in the expected volume, with the expected shape and distribution.

The crucial property is that it detects problems without a predefined rule for each one. It learns the normal behavior of a dataset and flags deviations from it. That is what lets it catch the failures teams did not foresee, which are precisely the ones that go unnoticed the longest.

The five pillars

Observability is usually organized around five signals, each answering a different question about a dataset's health.

• Freshness: is the data up to date? A table that normally updates hourly and has not changed in a day is a problem, even if every value in it is valid.
• Volume: did the expected amount of data arrive? A feed that usually brings a million rows and brought fifty thousand has failed silently upstream.
• Distribution: are the values within their normal range and shape? A sudden spike in nulls, a currency field that shifted by a factor of a hundred, a category that vanished, all signal something broke.
• Schema: did the structure change? A renamed column, a dropped field or a changed type can break consumers without any error in the pipeline itself.
• Lineage: when something is wrong, what upstream source caused it and what downstream reports are affected? Lineage turns an alert into a scoped incident.

Together these describe whether data is trustworthy right now, which is a different and broader question than whether it passes a fixed set of rules.

Observability versus quality rules

Data observability and a data quality framework are often confused, and the distinction is the whole point of having both.

Quality rules are assertions. You know a field must never be null, so you write a check, and it passes or fails. Rules encode what you already know matters. They are precise and they catch the failures you anticipated.

Observability is detection. It watches the data's own behavior and flags what deviates from the historical norm, including failures you never imagined and so never wrote a rule for. It is broader and fuzzier, and it catches the unknown unknowns.

Neither replaces the other. Rules without observability leave you blind to novel failures. Observability without rules leaves you with no defined standard, only deviation from the past, which is not the same as correct. A mature setup runs explicit rules for the requirements it knows and observability for everything it does not, and ties both to the same lineage so an alert points at its cause and its blast radius.

Incidents and time to resolution

The value of observability is measured in incidents caught early. Without it, a data incident is discovered by a consumer, a business user who notices the number looks wrong, which means it is found late, reported through the wrong channel, and diagnosed from scratch.

With observability, the incident is detected at the source, routed to the owning team, and scoped by lineage before anyone downstream is affected. The relevant metric is time to detection and time to resolution: how long a bad dataset sits undetected, and how long it takes to trace and fix once found. Cutting the weeks-of-silent-wrongness case down to a same-day alert is where observability earns its cost, because the damage from bad data scales with how long it goes unnoticed.

Keeping it actionable: the alert-fatigue trap

Observability has one dominant failure mode, and it is not technical. It is alert fatigue. Monitor every table with equal sensitivity and the team is buried in notifications, most of them noise, and learns to ignore all of them, including the one that mattered.

Avoiding this is a matter of discipline, not tooling:

• Tie monitors to consumers. A table that feeds the board report and the billing system warrants tight monitoring; a scratch table nobody depends on does not.
• Assign severity. Not every anomaly is an incident. Distinguish a warning from a page.
• Route to owners. An alert with no accountable owner is noise by construction. It has to reach the team that can act.
• Tune over time. Thresholds that fire constantly are wrong and must be adjusted, or they train the team to disbelieve them.

Tools like Monte Carlo, Soda, Bigeye and the open-source Elementary provide the machinery. Whether it stays useful depends on this curation, which is why observability, like the rest of a data platform, is an operating practice and not a purchase.

Failure patterns

From the field, the patterns that undermine data observability:

1. Observability as a replacement for rules. Anomaly detection bought as if it defined correctness, when it only detects deviation from the past.

2. Every table, equal weight. No prioritization by consumer, producing alert fatigue that buries the signal.

3. Alerts without owners. Notifications that reach no one accountable, so nothing is acted on.

4. No lineage. Anomalies detected but not traceable, so each alert triggers a diagnosis from scratch.

5. Untuned thresholds. Monitors that cry wolf until the team disables them, taking the useful alerts down with the noise.

Talk through your data observability

DNA Solutions helps European enterprises make data reliable in production: observability across freshness, volume, distribution, schema and lineage, combined with explicit quality rules and tied to ownership so an alert becomes a scoped incident, not noise. Whether dashboards are going wrong for weeks before anyone notices or alerts have become background static, we build observability that stays actionable. Talk to us.