About

Hi, I'm Alex Havrilla! I'm a research scientist at Google DeepMind studying the theoretical/practical limits of AI creativity and discovery. This work often spans a broad set of fields including Network Statistical and Approximation theory, RL, LLMs, Synthetic Data, and open-endedness.

I recently obtained my PhD in Machine Learning from Georgia Tech with a disseretation titled "Toward a Theory and Practice of Open-ended Reasoning with Generative Models". I was advised by (the amazing) Wenjing Liao. Also during this time I was fortunate to intern with FAIR, Microsoft Research, and Google Research. Additionally, I co-founded CarperAI: an early open-source research group studying RLHF at scale.

Previously, I graduated from Carnegie Mellon University with a joint MS/BS in mathematics and an additional major in computer science. My master's thesis surveyed and proved novel Khintchine type inequalities for random variables.

Research

My research covers a broad range of topics intersecting with generative modeling including LLMs, Reinforcement Learning, Diffusion Models, and statistical/approxmation theory for generative models. Recently, I’ve been deeply engaged in exploring ways to enhance the reasoning capabilities of Language Models (LLMs) for knowledge discovery, utilizing techniques from reinforcement learning. On the theoretical front, I focus on approximation and statistical theories for generative models, with a strong emphasis on validating these theories through empirical observations.

Selected papers

A. Havrilla, E. Hughes, M. Samvelyan, J. Abernethy, SPARQ: Synthetic Problem Generation for Reasoning via Quality-Diversity Algorithms. ArXiv. [pdf]

A. Havrilla, A. Dai, L. O’Mahony, K. Oostermeijer, V. Zisler, A. Albalak, F. Milo, S. Raparthy, K. Gandhi, B. Abbasi, D. Phung, M. Iyer, D. Mahan, C. Blagden, S. Gureja, M. Hamdy, W. Li, G. Paolini, P. Ammanamanchi, E. Meyerson, Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models. ArXiv. [pdf]

A. Havrilla, S. Raparthy, C. Nalmpantis, J. Yu, M. Zhuravinskyi, E. Hambro, R. Railneau, GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinements. Accepted to ICML 2024. [pdf]

A. Havrilla, W. Liao, Predicting Scaling Laws with Statistical and approximation Theory for Transformer Neural Networks. Accepted to Neurips 2024. [pdf]

A. Havrilla, Y Du, S. Raparthy, C. Nalmpantis, J. Yu, M. Zhuravinskyi, E. Hambro, R. Railneau, Teaching Large Language Models to Reason with Reinforcement Learning. Submitted to Neurips 2024. [pdf]

Y. Du, A. Havrilla, R. Railneau, A study in RL for LLM reasoning. Accepted to Neurips 2023 ICBINB workshop. [pdf]

T. Sawada, D. Paleka, A. Havrilla, P. Vidas, A. Kranias, P. Tadepilli, A. Komatsuzaki, ARB: An Advanced Reasoning Benchmark for Large Language Modeling. To appear in Neurips 2023 MathAI workshop. [pdf]

A. Havrilla, M. Zhuravynski, A. Tiwari, J. Tow, E. Kim, Q. Anthony, S. Biderman, L. Castricato. trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback. To appear in EMNLP 2023. [pdf]

A. Havrilla, M. Iyer, Training Large Language Models with Noisy Algorithmic Chain of Thought. To appear in ICML 2023 worksohp on Symbolic and Data driven methods for reasoning in NLP. [pdf]

B. Dahal, A. Havrilla, M. Chen, T. Zhao, W. Liao, On Deep Generative models for Approximation and Estimation of Distributions on Manifolds. To appear in Neurips 2022. [pdf]

My Thesis on Sharp Khintchine type Inequalities and some New Results [pdf]