Oak Ridge National Laboratory researchers have created a tool that can help researchers and AI developers speed up the task of fine-tuning how their machine learning models are processing natural language data, resulting in more accurate data analysis.
ORNL’s Transformer Explainability and Exploration tool, or TX2, tackles the issue of better understanding how artificial intelligence arrives at the conclusions it does—referred to as explainable AI. The software provides researchers a set of tools to gain clearer insights into how the AI model is generating results. By having greater visibility into the AI decision-making process, scientists can screen data and algorithms for deviation and bias and boost the accuracy of their models’ outcomes.
“Explainable AI is about having an understanding of what the tool is teaching itself and how it makes a decision so you can trust it,” noted ORNL’s Scott Stewart. “There are a lot of tie-ins to explainable AI. “For example, how do you make sure that a model doesn’t contain a bias related to the data source? Say it was trained on Granny Smith apples, so it never recognizes that a red apple is an apple. By using explainable tools, you figure out that it has that problem.”
TX2 is open-source software available on github, and can be easily plugged in as a widget to the experimental environment many data scientists use, Jupyter Notebook—a popular open-source interactive computing product. TX2 was designed to work with a type of deep learning natural language processing model known as a transformer. Transformers are trained to understand language context by training on very large text datasets and are better able to understand a sentence, for instance, compared with earlier techniques that might have simply counted the number of times the same word appeared in text.
The tool was developed with funding from the U.S. Department of Energy’s National Nuclear Security Administration. That effort seeks advanced approaches to develop AI-powered technologies for nuclear proliferation detection. Key to this effort is the use of explainable AI to help design robust AI models and establish justifiable confidence in model results. For instance, AI technology developers can use TX2 to understand why specific natural language processing techniques might better automatically distinguish between highly-technical details, like different types of chemical processes. Developers can then build specialized AI models to analyze textual data related to nuclear science and technology.
TX2 also has wider applications and could be deployed, for instance, to better understand and refine AI models created to scan medical text to improve diagnostic capabilities.
“If there has been a heart problem diagnosed, you could use TX2 to see what kind of language the transformer is picking up on that indicates heart issues,” said ORNL co-developer Nathan Martindale.
“We’ve designed TX2 to be accessible to a wide variety of users and on any transformer,” Martindale added. “It’s for anyone who wants to avoid having to create their own explainability tool. Regardless of how you’ve set up your model, it can give you a high-level overview of how it’s performing.” TX2 is also easy to customize to various levels of complexity. “All in all, the visualizations involved in this are relatively simple, but they can be very effective,” Martindale said.
For people working on explainability for their transformer models, now they don’t have to build it themselves, Stewart said. “That saves time and money. As researchers we should be all about putting things like this together, so we benefit as a community rather than spending a few months doing it on your own.”
ORNL is managed by UT-Battelle for DOE’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. DOE’s Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.
