Scientific Machine Learning Surrogate Modeling By Ioannis Papapanagiotou, Roland V. Bumbuc, H. Ibrahim Korkmaz, Valeria Krzhizhanovskaya, Vivek M. Sheraton

From simulations to surrogates: Neural networks enhancing burn wound healing predictions

AI models accelerate burn wound healing predictions, with a Spatio-temporal Attention LSTM (STA-LSTM) showing superior accuracy in emulating complex biological simulations.

🎙️

Listen to the Paper Podcast

An AI-generated conversation about this research

0:00 / --:--
This figure provides a visual comparison of the five different neural network architectures evaluated in the paper. It illustrates the sequence of layers for each model: LSTM, C-LSTM, CT-LSTM, STA-LSTM, and PINN. A legend at the bottom defines the symbols used for various components, including input/output layers, LSTM layers, Convolutional 2D (Conv2D) layers, Fully Connected (FC) layers, attention (STA-LSTM) layers, activation functions (ReLU), and pooling layers. The schematics highlight the structural differences, from the relatively simple feedforward PINN to the more complex convolutional and temporal structures of C-LSTM and CT-LSTM.
Fig. 1. Schematic representation of neural network architectures.
Fig. 1. Schematic representation of neural network architectures.

Key Takeaways

  • Neural networks can serve as effective and computationally efficient surrogates for slow, complex Agent-Based Models (ABMs) used in burn wound healing research.
  • The Spatio-temporal Attention Long Short-Term Memory (STA-LSTM) architecture demonstrated the best overall performance, accurately predicting both the spatial distribution and temporal dynamics of key immune-system markers (cytokines).
  • While theoretically promising, incorporating physical laws via a Physics-informed Neural Network (PINN) did not outperform specialized recurrent architectures like STA-LSTM for this specific regression task.

TL;DR

Severe burns trigger a complex immune response that can lead to life-threatening complications. To understand this process, researchers use powerful but extremely slow computer simulations called Agent-Based Models (ABMs). The significant time and computational power required to run these detailed simulations make them impractical for real-time clinical applications, creating a need for faster predictive tools that can forecast the wound's healing trajectory. This study investigates using various neural networks as fast 'surrogates' to mimic the ABM's output. The researchers trained different architectures, including several types of Long Short-Term Memory (LSTM) networks and a Physics-Informed Neural Network (PINN), to predict cytokine levels in a burn wound. Their key contribution is demonstrating that a Spatio-temporal Attention LSTM (STA-LSTM) model most accurately and efficiently predicts the healing process, outperforming the other models.

Why Does It Matter?

This paper provides a valuable blueprint for overcoming the computational bottleneck of complex biological simulations. By showing that surrogate models, particularly STA-LSTM, can rapidly and accurately predict outcomes, it paves the way for integrating sophisticated models into time-sensitive research and potential clinical decision-support tools. This work accelerates the translation of computational insights into practical applications for complex biological processes like wound healing.

ABM Bottleneck

The core 'ABM Bottleneck' identified in the paper is the immense computational expense and time-consuming nature of high-fidelity agent-based models (ABMs). While ABMs offer unparalleled detail in simulating the complex immune response to burns, this computational burden makes them impractical for rapid clinical decision-making. The research directly confronts this limitation by developing various neural networks as lightweight surrogate models. These surrogates are trained to mimic the behavior of the complex ABM, enabling them to generate predictions for cytokine concentrations almost instantly. This approach effectively bypasses the simulation bottleneck, offering a pathway to translate the deep insights from detailed biological models into a computationally efficient and clinically viable tool for burn healing predictions.

NN Model Zoo

The paper's 'NN Model Zoo' represents a thoughtful, structured exploration of diverse neural network architectures designed to serve as fast surrogates for a complex agent-based burn wound model. This collection isn't arbitrary; it systematically investigates how different architectural components handle the intricate spatio-temporal data of cytokine dynamics. The zoo ranges from a standard LSTM baseline to hybrid models incorporating convolutions and, most effectively, attention mechanisms. The analysis crucially reveals that the Spatio-Temporal Attention LSTM (STA-LSTM) achieves the best overall statistical performance, underscoring the power of attention in deciphering complex biological patterns. However, the inclusion of a Physics-Informed Neural Network (PINN) introduces a fascinating dichotomy, trading some statistical accuracy for enforced physical consistency. This comparison insightfully demonstrates that there is no universal 'best' model; instead, it presents a nuanced trade-off between maximizing predictive accuracy and ensuring biophysical plausibility.

Spatial vs Stats

This study reveals a critical dichotomy in predictive modeling: achieving high statistical accuracy versus faithfully reproducing spatial patterns. While conventional metrics like R² and MSE gauge overall performance, they can mask a model's failure to capture the essential spatial distribution of biological markers like cytokines. The analysis shows that certain models might capture aggregate statistical trends but fail to generate accurate spatial maps, limiting their biological utility. Conversely, spatiotemporally-focused models might show promise spatially but lag in statistical rigor. The research champions the STA-LSTM architecture as a solution that bridges this divide, excelling in both statistical benchmarks and the reconstruction of complex spatial dynamics, thus offering a more complete and insightful surrogate for the intricate healing process.

Scalability Woes

The analysis uncovers significant scalability woes for certain neural network surrogates. While aiming to replace computationally heavy Agent-Based Models, the study reveals that models incorporating convolutional layers, specifically C-LSTM and CT-LSTM, fail disastrously at scale. They either crash during training on larger grids (500x500) or their accuracy plummets, indicating an inability to handle the increased complexity of spatial dependencies. This highlights a critical limitation, as the utility of a surrogate model is tied to its performance on realistic, high-resolution simulations. In stark contrast, the STA-LSTM and PINN architectures demonstrate superior scalability, maintaining high prediction accuracy. This finding is crucial, emphasizing that the architectural choice is paramount for creating practical, scalable models for complex biological systems.

Next: GRUs, PINNs

Considering the paper's analysis of surrogate models for burn wound healing simulations, a future direction centered on 'GRUs and PINNs' represents a highly strategic refinement. Investigating Gated Recurrent Units (GRUs) would be a logical next step to optimize computational efficiency. As a less complex alternative to the LSTMs tested, GRUs could offer a superior trade-off between predictive accuracy and reduced computational demand. Concurrently, advancing the Physics-Informed Neural Network (PINN) framework indicates a push towards enhanced scientific plausibility. Deeper integration of the underlying biological diffusion equations would create a more robust model that not only predicts temporal trends but also respects physical laws, improving its generalization and trustworthiness beyond mere data-fitting.

More Figures (4)
Fig. 2. Performance metrics for neural networks on test set (time t = 72 - 89 hr and t = 82 - 100 hr) for gridsize 250 × 250. The metrics are also summarised numerically in Table A.3.
Fig. 2. Performance metrics for neural networks on test set (time t = 72 - 89 hr and t = 82 - 100 hr) for gridsize 250 × 250. The metrics are also summarised numerically in Table A.3.
Fig. 3. Spatial plots for neural networks at time steps t = 72, 82, 92, and 100 showing cytokine (for all 6) concentration of actual vs predicted values in order of accuracy (top to bottom). The computed binary spatial accuracy per method is displayed on the right. The results are also summarised in Table A.5.
Fig. 3. Spatial plots for neural networks at time steps t = 72, 82, 92, and 100 showing cytokine (for all 6) concentration of actual vs predicted values in order of accuracy (top to bottom). The computed binary spatial accuracy per method is displayed on the right. The results are also summarised in Table A.5.
Fig. 4. Time-series of mean cytokine concentrations (IL-8, IL-6, and TGF) for C-LSTM, STA-LSTM, and PINN at 100 × 100 grid dimensions. Means and standard deviations for this time period are given in boxes. Those cytokines were specifically chosen to show the variability in IL-8 prediction, the systematic bias for TGF, and the poor prediction of STA-LSTM for IL-6 values. The best neural networks’ predictions for Mean and Std are highlighted in bold.
Fig. 4. Time-series of mean cytokine concentrations (IL-8, IL-6, and TGF) for C-LSTM, STA-LSTM, and PINN at 100 × 100 grid dimensions. Means and standard deviations for this time period are given in boxes. Those cytokines were specifically chosen to show the variability in IL-8 prediction, the systematic bias for TGF, and the poor prediction of STA-LSTM for IL-6 values. The best neural networks’ predictions for Mean and Std are highlighted in bold.
Fig. 5. Computed averages of neural networks’ accuracy vs grid dimensions. C-LSTM and CT-LSTM produce significantly worse accuracy at higher grid dimensions due to their architecture being optimised locally at lower grid space.
Fig. 5. Computed averages of neural networks’ accuracy vs grid dimensions. C-LSTM and CT-LSTM produce significantly worse accuracy at higher grid dimensions due to their architecture being optimised locally at lower grid space.

This summary was generated by AI and may contain minor errors. Please refer to the original paper for authoritative details.