Feature Engineering at Scale: Automating Machine Learning Readiness in Data Lakes

Abstract

Feature engineering remains one of the most critical and labor-intensive steps in machine learning development. This study proposes an automated feature engineering framework integrated directly into modern data lake architectures. The framework enables reusable, versioned, and auditable feature pipelines that improve model reproducibility and deployment velocity. Case studies illustrate how automation significantly reduces time-to-model while maintaining data quality standards.