The Science
Oak Ridge National Laboratory researchers developed an interpretable long short-term memory (iLSTM) network for time-series prediction. The iLSTM enables interpretation of each variable’s importance and of variable-wise temporal importance for target predictions by exploring internal network structures. The network first enables hidden states to encode individual variables, so the contribution from individual inputs to the prediction can be distinguished. The network then uses a mixture-attention mechanism to summarize the variable-wise hidden states and jointly learns the network parameters for prediction and the importance weights for interpretation.
The Impact
The proposed iLSTM not only results in accurate time-series prediction, but also interprets the relative importance of inputs to outputs and their timescales of influence. Scientific machine learning not only promises to improve predictions but, perhaps more importantly, enhance predictive understanding. The insight into dynamic systems provided by iLSTM opens the black box of the LSTM network and answers the question of why it works, which improves understanding and guides scientific model development.
PI/Facility Lead: Dan Lu
Funding: DnC2S project funded by DOE-ASCR; TES-SFA project funded by DOE-BER and the LDRD AI Initiative
Publication:
- Lu, D., Ricciuto, D., and Liu, S., “An interpretable machine learning model for advancing terrestrial ecosystem predictions.” Accepted in ICLR Conference AI for Earth and Space Sciences Topic. Source: https://ai4earthscience.github.io/iclr-2022-workshop/accepted
Summary
We apply an interpretable Long Short-Term Memory (iLSTM) network for land-atmosphere carbon flux predictions based on time series observations of seven environmental variables. iLSTM enables interpretability of variable importance and variable-wise temporal importance to the prediction of targets by exploring internal network structures. The application results indicate that iLSTM not only improves prediction performance by capturing different dynamics of individual variables, but also reasonably interprets the different contribution of each variable
to the target and its different temporal relevance to the target. This variable and temporal importance interpretation of iLSTM advances terrestrial ecosystem model development as well as our predictive understanding of the system.
