Researchers developed an unsupervised framework called MotifAE, based on a sparse autoencoder architecture, to make sequence patterns in the “black box” structure of protein language models interpretable. The addition of a smoothness loss markedly improved the identification of known functional motifs compared with the standard method.
MotifAE detected some structural domains alongside short motifs; feature activities were correlated with the importance of residues in domain functions. Matching with experimental data enabled the prediction of features linked to the folding stability of domains and fitness landscapes specific to stability.
Why it matters
This study offers a way to examine the patterns generated by protein language models not merely as prediction outcomes, but by relating them to functional and structural elements within proteins. This makes it possible to investigate the link between model features and the importance of specific residues, and to assess the biological meaning of model outputs more directly. The method’s ability to distinguish structural domains in addition to short motifs shows that interpretability is not limited to individual sequence segments. The matching established with experimental data, meanwhile, indicates that features linked specifically to folding stability and the corresponding fitness landscapes can be examined through the model. However, which different protein contexts the framework presented in the study can explain to the same extent remains an open research question.