Skip to main content

Overview

The Anomaly Detection transform identifies statistical outliers in numeric data columns. It supports multiple detection methods and configurable actions — from flagging anomalies for review to automatically removing or capping them. Available on Professional plans and above.

Detection Methods

Method Selection Guide

Configuration

Threshold Interpretation

Actions

Start with the Flag action to review detected anomalies before committing to removal or capping. Switch to a destructive action only after confirming the detection method and threshold match your data.

Examples

Financial Data Quality

Flag transactions with extreme amounts before loading to the warehouse:
Query flagged rows:

Sensor Data Cleaning

Cap extreme sensor readings to prevent dashboard spikes:

Rolling Window Detection

Detect anomalies based on recent trends rather than all-time statistics:

Pipeline Patterns

Review-then-Clean

Multi-Column Analysis

Apply detection to multiple numeric columns in one node:

Tips

  • Z-Score assumes normally distributed data — verify with Data Profiling first
  • IQR is more robust to extreme outliers than Z-Score because it uses quartiles rather than mean
  • Lower thresholds flag more anomalies (higher sensitivity, more false positives)
  • Window Size enables rolling statistics — useful for time-series data where the distribution shifts over time
  • Combine with Validation nodes for comprehensive quality: Anomaly Detection catches statistical outliers while Validation catches rule-based violations

Data Quality Nodes

All data quality transforms including validation and profiling

Data Contracts

Rule-based validation at the contract level

Observability

Monitor anomaly rates across pipeline runs

Column Transforms

Additional column-level operations