Physics-guided Neural Networks (PGNNs)
Physics-based models are at the heart of today’s technology and science. Over recent years, data-driven models started providing an alternative approach and outperformed physics-driven models in many tasks. Even so, they are data-hungry, their inferences could be hard to explain and generalization remains to be a challenge. Combining data and physics could reveal the best of both worlds.
When machine learning algorithms are learning, they are actually searching for a solution in the hypothesis space you defined by your choice of algorithm, architecture, and configuration. Hypothesis space could be quite large even for a fairly simple algorithm. Data is the only guide we use to look for a solution in this huge space. What if we can use our knowledge of the world — for example, physics— together with data to guide this search?
|Finally reached the food section (courtesy of Dominic L. Garcia).|
|(1) Feature engineering using a physics-based model|
|(2) Data + Physics driven loss function|
The first approach, feature engineering, is extensively used in machine learning. The second approach, however, is compelling. Very much like adding a regularization term to punish overfitting, they add a physical inconsistency term to the loss function. Hence, with this new term, the optimization algorithm should also take care of minimizing physically inconsistent results.
- Achieving generalization is a fundamental challenge in machine learning. Since physics models, mostly, do not depend on a specific dataset, they might perform well on unseen data, even from a different distribution.
- Machine learning models are sometimes referred to as black-box models due to the fact that it is not always clear how a model reaches a specific decision. There is quite a lot of work going into Explainable AI (XAI) to improve model explainability and interpretability. PGNNs could provide a basis for XAI.
|Physical inconsistency term of the loss function|
- PHY: General lake model (GLM).
- NN: A neural network.
- PGNN0: A neural network with feature engineering. Results of the GLM are fed into the NN as additional features.
- PGNN: NN with feature engineering and with the modified loss function.
- RMSE: Root mean square error.
- Physical Inconsistency: Fraction of time-steps where the model makes physically inconsistent predictions.
|Results on Lake Mille Lacs|
|Results on Lake Mendota|