Mechanistic Machine Learning

MInD Harvard Team Visit

George G. Vega Yon, Ph.D.

2023-05-19

Overview

Slides can be downloaded from
https://ggv.cl/slides/mind2023

Machine Learning is Broken

After all the data pouring, attention to causal inference and mechanistic models is coming back (Baker et al. 2018; Pearl 2019)

The case of Google Flu Trends: Paper reported a 0.97 correlation (Ginsberg et al. 2009), but predictions overshoot by 100% (Kandula and Shaman 2019; Lazer et al. 2014).

Mechanistic Machine Learning

Mechanistic Machine Learning [MechML]–a.k.a. theory-guided data science/machine learning: A hybrid between theory and data-driven prediction.

Mechanistic models

Inference-driven (causality).
Great for small datasets.
Knowledge beyond the observed data.

Machine Learning

Data-driven (prediction).
Great for big data.
Finds hidden knowledge in observed data.

ML can help explain what theory hasn’t… but we still need theory (Lazer et al. 2014)!

MechML: State-of-the-art

Adjusting errors in mechanistic-based prediction models (like ABMs). (Compagni et al. 2022)

Incorporating mechanistically inferred data as an additional -omics layer. (Zampieri et al. 2019)

Using pathway networks to add “external knowledge” as features. (Al taweraqi and King 2022)

Creating a loss function with a mechanistic penalty for modeling tumor cell density (Gaw et al. 2019)

Using simulations to inform neural networks for epidemic forecasting (Wang, Chen, and Marathe 2020).

and more (Jorner et al. 2021; Willard et al. 2022; Jia et al. 2021; von Rueden et al. 2023)

Warning

Mechanistic Machine Learning is not domain-knowledge-aided feature engineering. You need a whole other model to complement the ML algorithm.
This isn’t just an ML ensemble; you must have an ML and a Mech model.

How to “MechML”?

MechML: Three strategies

ML Correction: Use machine learning to learn the errors of a mechanistic model.

Mechanistic Feature: Add mechanistic predictions as a feature of a machine learning model.

Mechanistic Penalty: Add constraints to the ML algorithm based on a mechanistic model.