Cross-Source ML Model Training

Machine learning (ML) often operates on data fragmented across silos through two paradigms: distributed or centralized. This study illuminates the underexplored signifi-cance of data integration (DI) metadata in both methodologies. Our contribution is threefold. First, we formalize the complex relationships of data sources with DI metadata. Second, we propose an approach that transforms DI metadata into matrix representations, and streamlines data transformation and linear algebra operations over source datasets. Third, we present an optimization method, effectively deciding between factorization versus materialization. By leveraging logic-based pruning rules and an ML-based cost estimator, our approach outperforms state-of-the-art baselines and makes the trade-off of factorization and materialization with up to 90.5% accuracy.