Reinforcement Learning from Verifiable Rewards (RLVR) has been able to improve correctness in LLMs for domains like math and coding. But for many scientific problems, like materials science, there is no single "correct" result. You want many candidates that are all thermodynamically stable, distinct from each other, and outside our current knowledge of crystal space.
We built PLaID++, an LLM post-trained for multi-objective property-guided crystal generation, to solve this problem.
We introduce Reinforcement Learning from Interatomic Potentials, a diversity-aware framework for materials discovery
Motivated by mode collapse on a naïve 3D representation, we develop a symmetry-informed encoding to scale reinforcement learning
We show generalization across multiple materials discovery objectives, and achieve SOTA results in all settings
The Setup
We start from Qwen-2.5 7B and fine-tune it with LoRA on MP-20, ~45k inorganic crystals from the Materials Project. To guide our search space towards chemically useful structures, we propose RLIP (Reinforcement Learning from Interatomic Potentials). The reward comes from EquiformerV2, a machine-learned interatomic potential that predicts a structure's energy, a (rough) proxy for real world synthesizability, allowing us to rank candidates by thermodynamic stability and run iterative DPO on preference pairs.
Our primary metric is the S.U.N. rate: the percentage of generated crystals that are simultaneously stable, unique, and novel. We evaluate across three settings — unconditional generation, space-group-conditioned generation (scored by S.S.U.N., which additionally requires the generated structure to satisfy the requested symmetry), and high bulk modulus-conditioned generation (which requires the structure to have hardness ≥ 325 GPa).
Mode Collapse
The obvious first attempt is to RL the LLM using a standard 3D coordinate representation like prior work, where each fractional coordinate exists on a seperate line per atom. However, interestingly, doing this leads novelty to fall drastically while stability keeps climbing across multiple iterations. The model games the reward by memorizing a few stable structures and emitting them over and over.
Wyckoff Representation
Crystal structures inherently follow symmetry constraints that can be represented by space groups and Wyckoff positions. A crystals symmetry is directly correlated with downstream properties like piezoelectricity. We encode each crystal as a compact, Wyckoff text representation, which vastly improves performance under RLIP. We hypothesize that our Wyckoff representation works because of its higher information density, where local atom type or coordinate changes propagate to globally meaningful structure differences, expanding the diversity of valid structures the model can learn and generate. Below are a few structures you can click through to directly compare how our Wyckoff representation compares to prior 3D coordinate methods.
Temperature as an Entropy Regularizer
Representation determines what the model can express; sampling determines how much of that space it explores. The most consequential sampling parameter for crystal diversity is temperature. In the context of materials discovery, we find increasing temperature across successive iterations of RLIP is critical to maintains exploration of novel chemical space and avoid overoptimization of stability.
Moderate temperature acts as an entropy regularizer — broad coverage while staying inside the stable region.
SOTA Results
Combining the Wyckoff representation, RLIP, and the dynamic temperature schedule, PLaID++ reaches a 22.3% stability rate and a 7.74% S.U.N. rate on unconditional MP-20 generation — roughly 50% higher S.U.N. than the best prior method — without any specialized equivariant architecture:
Cross Task Transfer
Running DPO on space-group-conditioned pairs alone lifts conditional S.S.U.N. by 22% over the base Wyckoff model. Mixing in preference pairs from unconditional generation — a different task — more than doubles that improvement. The gains are most pronounced exactly where conditional training data is scarce
The improvements hold out of distribution. We held eight space groups out of the RLIP reward entirely, then prompted the model to generate structures for them. PLaID++ outperforms fine-tuning alone by an average of 20% S.S.U.N. on these held-out groups. This suggests the model learns structural principles that apply across symmetry groups rather than memorizing the reward distribution.
Closing Thoughts
Over the past few months, I've seen a surge in interest towards applications of AI for materials from the broader AI research community. This is incredibly exciting, and I believe that many of the results r.e. better representations and cross-task transfer are promising signals for the viability of foundational agents/models in this domain.
The full details, ablations, and benchmarks are in the paper, the code is on GitHub, and the weights are on HuggingFace.
Feel free to reach out if you found this work interesting :)