Machine learning models for physics and engineering

The main objective of physics is to understand nature, and one way to achieve this is to build models of the natural phenomenon of interest. For centuries, physicists have been pushing the limits of our knowledge, and in recent years we have commonly begun to encounter challenges where our physics-driven models are either not accurate enough, or too complex, to be useful. In these situations, machine learning models can give a helping hand. 

A model is a representation of some relevant parts of reality. A statue is a model of a person, a word is a model of a thing. In physics, we are interested in mathematical models – equations describing the world. These models help us understand phenomena and predict their behavior. Models are a simplification of reality, based on assumptions. As long as the assumptions are sound, model predictions are expected to be accurate. Force = mass x acceleration is a famous model that works very well for many everyday scenarios but fails for tiny objects (electrons), massive objects (black holes), or fast objects (photons).

Accuracy and simplicity are the two important factors for evaluating models. Most of the time there is a trade-off between the two. A city map drawn to a scale of 1:1 can be very accurate but very complex, hence totally useless. Therefore for a model to be useful, it should be above a certain accuracy and below a certain complexity where the specifications depend on the application.

As an example, let's look at 3 models for the shape of our Earth: Flat, Sphere, Oblate spheroid (Figure 1). As accuracy increases, the computational complexity of the model increases.

Figure 1 Three different models for the shape of the Earth.

Choosing which model to use depends very much on the application. Even though we are not "flat earthers", we actually consider the Earth as "flat" for many of our day-to-day activities such as going to the supermarket. But for many other activities like launching a satellite, we need more accurate models and we can afford more computational complexity. Our application specifies the shaded blue area we want our models to be in (Figure 2).

Figure 2  Accepted mod el spaces (shaded blue areas) for two different applications. Note that for going to the supermarket, the constraint is an upper bound on computational complexity, while for launching a satellite, the constraint is a lower bound on accuracy.

We frequently experience situations where our physics models are either not accurate enough (Model X, Fig. 3) or computationally too complex (Model Y, Fig. 3) to satisfy our application (Blue shaded area, Fig. 3). Machine learning can help in two fundamental ways (Figure 3):  (1) It can improve accuracy, without increasing the complexity. (2) It can decrease complexity without a loss of accuracy.  Let's see some examples for both.

Figure 3  Two physics-based models X and Y out of specification. Two ways machine learning can help (1) by increasing accuracy or (2) by decreasing computational complexity.

In one of our recent projects on building a novel particle size analyzer, the particle scattering model based on Mie theory was not accurate enough for the application. Using a machine learning model (random forests) we were able to build a model for particle size prediction that satisfied the specifications. You can read more here in our paper. In this case, we were able to increase model accuracy without increasing computational complexity (Figure 3, (1)). In another project related to microscopy, the image reconstruction algorithm based on physical diffraction laws took around 10 minutes to run, hindering the ability of the microscope to provide "live" images. We helped our collaborators by developing a deep learning model that can achieve the same construction in less than 1 second.  Here, we were able to simplify the computational complexity of the calculation, while not sacrificing any accuracy (Figure 3, (2)).

 

But how can machine learning achieve these impressive results? As we discussed, every model requires assumptions. Often, the more/strong assumptions you make, the more you lose accuracy. Physical models require some strong assumptions (no multiple scattering, the particles are perfect spheres, etc.) whereas for building machine learning models you make weaker assumptions (this random forest has enough capacity to learn the scattering relationship from this dataset). As for the computational complexity, of course, it does not vanish. Rather you push it to the training phase of machine learning model development. In other words, creating the model might take time, but once the model is trained and ready to use, it generates new predictions very quickly.

 

Machine learning has the potential to transform many areas: from sensing to materials discovery, to imaging, to optimal control of dynamic systems. If you are interested in improving your models through machine learning get in touch with us!