An artificial intelligence was able to produce accurate predictions about the composition of a metal alloy, but used information with no physical relevance to reach the results. The discovery was presented by researchers from Case Western Reserve University in a study published on Monday (19) in the scientific journal Communications Materials.
The research demonstrated that a neural network can show high statistical accuracy without identifying the physical signals it should analyze. The problem was detected in a machine learning system applied to X-ray diffractograms, records used to investigate the structure of materials.
The scientists analyzed the ability of two neural networks to estimate the proportions of the alpha and beta crystalline phases present in the Ti–6Al–4V titanium alloy. Although both achieved similar results, the mechanisms used to make the predictions were fundamentally different.
Two accurate models, but only one learns the physical signal
The first model, called IE, had received initial training with natural images from the ImageNet database. The second, called ISE, also went through a training stage with simulated diffractograms. Subsequently, both were fine-tuned with only 250 experimental diffractograms.
In conventional tests, both networks showed high accuracy. However, robustness assessments and attention maps revealed a decisive difference.
The IE model had learned to use background textures and pixel noise, characteristics that do not directly represent the physical properties investigated. The ISE, in turn, concentrated its predictions on the Bragg reflections, diffraction patterns associated with the crystalline structure of the material.
This means that the first system was able to produce seemingly reliable answers from information unsuitable for the task, while the second used physically relevant evidence.
The work, led by Redad Mehdi and collaborators, shows that evaluating only the accuracy of predictions may be insufficient to validate scientific AI models. The results refer specifically to the analysis of diffractograms and indicate the importance of incorporating physical knowledge into training and testing the robustness of systems.
