Diego Calanzone

I am a Ph.D. student at Mila Quebec AI Institute under the supervision of Prof. Pierre-Luc Bacon. Currently, my research consists of Reinforcement Learning with Large Language Models for applications (drug discovery, material discovery) and theory (hierarchical teinforcement Learning, generative rewards).

With my studies in CS/AI, I developed a background in software engineering, logic, computer vision and language. Deep research questions lie at the intersection: what priors are necessary and sufficient to develop intelligent behavior? How to learn efficiently? What are the benefits for humanity in scaling artificial intelligence?

In this web space I aim to share my research in the broader sense, that is the pursuit of understanding of life, intelligence and creativity. My interests are highly heterogenous, from social psychology to environmental activism and analog life. In my spare time, I play music, enjoy board sports and read mostly about research in other fields.

news

Jan 11, 2025	Our work Discovery of Sustainable Refrigerants through Physics-Informed RL Fine-Tuning of Sequence Models has been accepted to the SIMBIOCHEM Workshop @ EurIPS 2025!
Jan 10, 2025	We presented Logically Consistent Language Models via Neuro-Symbolic Integration as a full paper @ ICLR 2025!
Jan 10, 2025	Our work Mol-MoE: Training Preference-Guided Routers for Molecule Generation has been admitted to the GEM Bio Workshop @ ICLR 2025!
Oct 16, 2024	Our paper Logically Consistent Language Models via Neuro-Symbolic Integration has been accepted at the System 2 Reasoning At Scale Workshop @ NeurIPS 2024! 🥳
Aug 1, 2024	I will serve as tutor and creator for the RL Lab @ M2L Summer School 2024 in Milan! Learning contents to be released soon.

Publications

Reasoning

Logically Consistent Language Models via Neuro-Symbolic Integration

Diego Calanzone, Stefano Teso, and Antonio Vergari

NeurIPS 2024 Workshop on System 2 Reasoning at Scale

Abs

Large language models (LLMs) are a promising venue for natural language understanding and generation. However, current LLMs are far from reliable: they are prone to generating non-factual information and, more crucially, to contradicting themselves when prompted to reason about relations between entities of the world. These problems are currently addressed with large scale fine-tuning or by delegating reasoning to external tools. In this work, we strive for a middle ground and introduce a loss based on neuro-symbolic reasoning that teaches an LLM to be logically consistent with an external set of facts and rules and improves self-consistency even when the LLM is fine-tuned on a limited set of facts. Our approach also allows to easily combine multiple logical constraints at once in a principled way, delivering LLMs that are more consistent w.r.t. all constraints and improve over several baselines w.r.t. a given constraint. Moreover, our method allows LLMs to extrapolate to unseen but semantically similar factual knowledge, represented in unseen datasets, more systematically.
AI4Science

Discovery of Sustainable Refrigerants through Physics-Informed RL Fine-Tuning of Sequence Models

Adrien Goldszal, Diego Calanzone, Vincent Taboga, and 1 more author

arXiv preprint arXiv:2509.19588

Abs

We propose a method to discover environmentally sustainable refrigerant molecules by combining physics-informed predictors and reinforcement learning fine-tuning of sequence models. Our approach constrains the search via thermodynamic, safety, and environmental criteria, filtering candidate molecules to only those satisfying multiple rigorous properties. This enables identification of promising refrigerants despite scarce ground-truth data, and supports generalization beyond the training set.
AI4Science

Mol-MoE: Training Preference-Guided Routers for Molecule Generation

Diego Calanzone, Pierluca D’Oro, and Pierre-Luc Bacon

arXiv preprint arXiv:2502.05633

Abs

Recent advances in language models have enabled framing molecule generation as sequence modeling. However, existing approaches often rely on single-objective reinforcement learning, limiting their applicability to real-world drug design, where multiple competing properties must be optimized. Traditional multi-objective reinforcement learning (MORL) methods require costly retraining for each new objective combination, making rapid exploration of trade-offs impractical. To overcome these limitations, we introduce Mol-MoE, a mixture-of-experts (MoE) architecture that enables efficient test-time steering of molecule generation without retraining. Central to our approach is a preference-based router training objective that incentivizes the router to combine experts in a way that aligns with user-specified trade-offs. This provides improved flexibility in exploring the chemical property space at test time, facilitating rapid trade-off exploration. Benchmarking against state-of-the-art methods, we show that Mol-MoE achieves superior sample quality and steerability.