Machine Learning-Driven Anomaly Detection for Data Pipeline Reliability

Abstract

Ensuring reliability of data pipelines is essential for mission-critical analytics and machine learning workloads. This research investigates machine learning-based anomaly detection techniques applied to pipeline metrics and metadata. Results show that predictive monitoring significantly reduces downtime and manual intervention.