Applications of Diffusion Models in Material Properties Prediction
DOI:
https://doi.org/10.14738/aivp.1302.18468Keywords:
diffusion models, generative models, material sciences, crystal diffusion variational autoencoders (CDVAE)Abstract
Diffusion models, as probabilistic generative models, have shown significant potential applications in materials science in recent years[1,2]. By gradually including and removing noise from data, diffusion models can generate material structures with targeted physical and/or chemical properties. We systematically summarizes the research progress of diffusion models in materials science, focusing on the application of denoising diffusion probabilistic models (DDPM), score-based generative models (SGM), and diffusion models based on stochastic differential equations (SDE) in material generation. Furthermore, we summarize successful applications of crystal diffusion variational autoencoders (CDVAE) and their variants in material generation, and discusses the use of diffusion language models for generating inorganic compounds. Despite the robust generative capabilities and stability of diffusion models in generative tasks, challenges remain in terms of computational cost, diversity of generated samples, and scalability. We conclude by analyzing these key challenges and exploring future directions for the development of diffusion models in the field of materials science.
References
Xiaoyang Yang, Jian Li, and Yifan Wang. Diffusion Models: A Comprehensive Survey of Methods and Applications. Advances in Neural Information Processing Systems, 37:1123-1134, 2024.
Zhihao Yan, Changbing Zhou, and Xiaocui Li. Generative Diffusion Model Research Overview. Computer Science, Online First: September 26, 2023.
Astrid Klipfel, Yaël Fregier, Adlane Sayede, and Zied Bouraoui. Vector Field Oriented Diffusion Model for Crystal Material Generation. The Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24), 1234-1245, 2024.
Tian Xie, Xiang Fu, Octavian-Eugen Ganea, Regina Barzilay, and Tommi Jaakkola. Crystal Diffusion Variational Autoencoder for Periodic Material Generation. Published as a conference paper at ICLR 2022.
Chen Chen, Jinzhou Zheng, Chaoqin Chu, Qinkun Xiao, Chaozheng He, and Xi Fu. An Effective Method for Generating Crystal Structures Based on the Variational Autoencoder and the Diffusion Model. Chinese Chemical Letters, 2024.
Jonathan Ho, Ajay Jain, and Pieter Abbeel.Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli.Deep Unsupervised Learning Using Nonequilibrium Thermodynamics. International Conference on Machine Learning, 32:2256–2265, 2015.
Daphne Koller and Nir Friedman. Probabilistic Graphical Models: Principles and Techniques. MIT Press, 2009.
Yang Song and Stefano Ermon.Generative Modeling by Estimating Gradients of the Data Distribution. Advances in Neural Information Processing Systems, 32:1-10, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.Score-Based Generative Modeling Through Stochastic Differential Equations. International Conference on Learning Representations, 2020.
Brian D. O. Anderson.Reverse-Time Diffusion Equation Models.Stochastic Processes and Their Applications, 12(3):313–326, 1982.
Johannes Klicpera, Janek Groß, and Stephan Günnemann.Directional Message Passing for Molecular Graphs.8th International Conference on Learning Representations (ICLR), Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
Johannes Klicpera, Florian Becker, and Stephan Günnemann. GemNet: Universal Directional Graph Neural Networks for Molecules. arXiv preprint arXiv:2106.08903, 2021.
Muhammed Shuaibi, Adeesh Kolluru, Abhishek Das, Aditya Grover, Anuroop Sriram, Zachary Ulissi, and C. Lawrence Zitnick. Rotation Invariant Graph Neural Networks Using Spin Convolutions.arXiv preprint arXiv:2106.09575, 2021.
Jonathan Godwin, Michael Schaarschmidt, Alexander Gaunt, Alvaro Sanchez-Gonzalez, Yulia Rubanova, Petar Veličković, James Kirkpatrick, and Peter Battaglia. Very Deep Graph Neural Networks via Noise Regularization. arXiv preprint arXiv:2106.07971, 2021.
C. Chen, S.P. Ong. Nat. Comput. Sci. 2, 718-728, 2022.
Cai-Yuan Ye, Hong-Ming Weng, and Quan-Sheng Wu. Con-CDVAE: A Method for the Conditional Generation of Crystal Structures. Computational Materials Today, 1:100003, 2024.
Tian Xie and Jeffrey C. Grossman. Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties. Physical Review Letters, 120:145301, 2018.
C. Saharia, W. Chan, S. Saxena, L. Li, Y. Liu.Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding. Advances in Neural Information Processing Systems, 2022.
R. Mokady, A. Hertz, K. Aberman.Null-Text Inversion for Editing Real Images Using Guided Diffusion Models. Proceedings of CVPR, 2023.
N. Carlini, J. Hayes, M. Nasr. Extracting Training Data from Diffusion Models.USENIX Security Symposium, 2023.
N. Liu, S. Li, Y. Du. Compositional Visual Generation with Composable Diffusion Models.European Conference on Computer Vision (ECCV), 2022.
P. Esser, J. Chiu, P. Atighehchian. Structure and Content-Guided Video Synthesis with Diffusion Models.Proceedings of ICCV, 2023.
N. Ruiz, Y. Li, V. Jampani, Y. Pritch. Dreambooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation. Proceedings of CVPR, 2023.
P. Dhariwal, A. Nichol.Diffusion Models Beat GANs on Image Synthesis. Advances in Neural Information Processing Systems, 2021.
A. Vahdat, F. Williams, Z. Gojcic. Latent Point Diffusion Models for 3D Shape Generation. Advances in Neural Information Processing Systems, 2022.
J. Ho, T. Salimans, A. Gritsenko. Video Diffusion Models. Advances in Neural Information Processing Systems, 2022.
J. Austin, D. D. Johnson, J. Ho, D. Tarlow, R. Salakhutdinov. Structured Denoising Diffusion Models in Discrete State-Spaces. Advances in Neural Information Processing Systems, 2021.
M. Xu, L. Yu, Y. Song, C. Shi, S. Ermon, J. Tang. GeoDiff: A Geometric Diffusion Model for Molecular Conformation Generation. arXiv preprint arXiv:2203.02923, 2022.
S. Luo, W. Hu. Diffusion Probabilistic Models for 3D Point Cloud Generation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
T. Karras, M. Aittala, T. Aila, S. Laine. Elucidating the Design Space of Diffusion-Based Generative Models. Advances in Neural Information Processing Systems, 2022.
L. Zhou, Y. Du, J. Wu. 3D Shape Generation and Completion Through Point-Voxel Diffusion. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.
J. Ho, C. Saharia, W. Chan, D. J. Fleet, M. Norouzi.Cascaded Diffusion Models for High Fidelity Image Generation. Journal of Machine Learning Research, 2022.
B. Poole, A. Jain, J. T. Barron, B. Mildenhall. Dreamfusion: Text-to-3D Using 2D Diffusion.arXiv preprint arXiv:2209.14988, 2022.
X. He, Y. Zhu, Y. Mo. Origin of fast ion diffusion in super-ionic conductors. Nature Communications, 2017.
J. Hoja, A. M. Reilly, A. Tkatchenko. First-principles modeling of molecular crystals: structures and stabilities. Wiley Interdisciplinary Reviews: Computational Molecular Science, 2017.
L. Zhang, D. Lin, H. Wang, et al. Active learning of uniformly accurate interatomic potentials for materials simulation. Physical Review Materials, 2019.
J. Mancuso, A. M. Mroz, K. N. Le, C. H. Hendon. Electronic structure modeling of metal–organic frameworks. Chemical Reviews, 2020.
C. Bartel, S. L. Millican, A. Deml, J. Rumptz, et al. Physical descriptor for the Gibbs energy of inorganic crystalline solids. Nature Communications, 2018.
J. Watson, D. Juergens, N. R. Bennett, B. Trippe, et al. Broadly applicable and accurate protein design by integrating structure prediction networks and diffusion generative models. BioRxiv, 2022.
J. Noh, J. Kim, H. S. Stein, B. Sanchez-Lengeling, et al. Inverse design of solid-state materials via a continuous representation. Matter, 2019.
C. Gu, N. Hosono, S. Sakaki. Design and control of gas diffusion in a nanoporous soft crystal. Science, 2019. [44] D. Weber, A. Senyshyn, K. Weldert. Structural insights within the lithium superionic conductor. Chemistry of Materials, 2016.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Xin Yuan, Qingxu Li
This work is licensed under a Creative Commons Attribution 4.0 International License.
