Leon Li - 李昂

al4263[at]columbia[dot]edu

I am a Master’s student in Computer Science at Columbia University, where I am fortunate to be advised by Prof. Micah Goldblum, Prof. Tianyi Peng, Prof. Jing Dong, and Prof. Hongseok Namkoong. Previously, I graduated magna cum laude from Columbia University with a B.S. in Computer Science in 2024.

My research goal is to develop AI systems and algorithms that can push the boundaries of reasoning and decision-making, while ensuring alignment and safety. Specifically, my interests include:

Reasoning and Decision Making: Reinforcement Learning, Exploration, Uncertainty Quantification, Bayesian Inference, Meta-Learning, Test-Time Algorithm, In Context Learning, LLM Reasoning, and Foundation Models for Decision Making

AI Alignment and Safety: Online Alignment, Self-Alignment, Weak-to-Strong Alignment, Personalization, AI Safety.

AI + X: Applications in Operations Research, Societal Systems, and Scientific Discovery

Work in Progress and Preprints

2025

Efficient Exploration for Online RLHF

2025

In preparation for ICML 2025

AI Alignment Exploration Uncertainty Quantification

Abs

RLHF-UQ
Separating Aleatoric and Epistemic Uncertainty Through Joint Prediction with Exchangeable Sequence Models

2025

In preparation for ICML 2025

Bayesian Inference In Context Learning

Abs

Separating Aleatoric and Epistemic Uncertainty Through Joint Prediction with Exchangeable Sequence Models
Taxonomy of LLM based Agent Safety

2025

In preparation for ICML 2025

AI Agent AI Safety

Abs

Taxonomy of LLM based Agent Safety

2024

Generative Bias: LLM Generated Personas Lead to More Biased Simulations

2024

In preparation for ICML 2025

LLM Driven Simulation LLM Bias

Abs

Generative Bias

Publications

2024

QGym: Scalable Simulation and Benchmarking of Queuing Network Controllers

Haozhe Chen^*, Ang Li^*, Ethan Che^*, Tianyi Peng, Jing Dong, and Hongseok Namkoong

In Advances in Neural Information Processing Systems 37, Datasets and Benchmark Track, 2024

RL Model-Based Learning

Abs arXiv Code

Queuing network control determines the allocation of scarce resources to manage congestion, a fundamental problem in manufacturing, communications, and healthcare. Compared to standard RL problems, queueing problems are distinguished by unique challenges: i) a system operating in continuous time, ii) high stochasticity, and iii) long horizons over which the system can become unstable (exploding delays). To spur methodological progress tackling these challenges, we present an open-sourced queueing simulation framework, QGym, that benchmark queueing policies across realistic problem instances. Our modular framework allows the researchers to build on our initial instances, which provide a wide range of environments including parallel servers, criss-cross, tandem, and re-entrant networks, as well as a realistically calibrated hospital queuing system. QGym makes it easy to compare multiple policies, including both model-free RL methods and classical queuing policies. Our testbed complements the traditional focus on evaluating algorithms based on mathematical guarantees in idealized settings, and significantly expands the scope of empirical benchmarking in prior work.
PersonalLLM: Tailoring LLMs to Individual Preferences

Thomas Zollo^*, Andrew Siah^*, Naimeng Ye, Ang Li, and Hongseok Namkoong

Pluralistic Alignment at NeurIPS 2024 Workshop, In submission to ICLR 2025, 2024

AI Alignment Personalization

Abs arXiv Code

As LLMs become capable of complex tasks, there is growing potential for personalized interactions tailored to the subtle and idiosyncratic preferences of the user. We present a public benchmark, PersonalLLM, focusing on adapting LLMs to provide maximal benefits for a particular user. Departing from existing alignment benchmarks that implicitly assume uniform preferences, we curate open-ended prompts paired with many high-quality answers over which users would be expected to display heterogeneous latent preferences. Instead of persona-prompting LLMs based on high-level attributes (e.g., user’s race or response length), which yields homogeneous preferences relative to humans, we develop a method that can simulate a large user base with diverse preferences from a set of pre-trained reward models. Our dataset and generated personalities offer an innovative testbed for developing personalization algorithms that grapple with continual data sparsity–few relevant feedback from the particular user–by leveraging historical data from other (similar) users. We explore basic in-context learning and meta-learning baselines to illustrate the utility of PersonalLLM and highlight the need for future methodological development.

Research Projects

2023

Novel Memory Plasticity and Readout Methods for Continual Scene Detection

Ang Li

Work done as a research assistant at Center for Theoretical Neuroscience, Zuckerman Institute at Columbia , 2023

AI for Neuroscience Neural Network Understanding

Abs PDF Code

In this paper, we introduce novel memory plasticity mechanisms called hybrid memory plasticity that can combine any sequences of multiplicative and additive plasticities, alongside innovative readout layers for effective memory storage retrieval, for addressing tasks that require temporal integration and classification. Our approach is applied to a continual scene detection task, where each scene comprises multiple frames requiring both memory integration and accurate classification. We have shown that our new biologically plausible neural network achieves SoTA result on the task we defined. Furthermore, we conduct a detailed neural network interpretability analysis to elucidate the mechanisms underlying the proposed memory plasticities. This framework provides valuable insights into biologically inspired architectures and their applications in continual learning, offering a deeper understanding of neuroscience principles.

2022

Analysis of ELBO loss landscape

Ang Li^*, and Xinyu Wei^*

2022

Neural Network Understanding

Abs PDF

We present a comprehensive empirical study of the Evidence Lower Bound (ELBO) landscape in variational inference, addressing fundamental questions about the geometric properties of its local maxima and their connectivity. While prior work has suggested the existence of Maximum Energy Paths (MEPs) connecting local optima in the ELBO parameter space, systematic empirical validation remains incomplete. We adapt the string method to construct MEPs and introduce a novel visualization approach using random vector projection to analyze the ELBO landscape. Through extensive experiments on Latent Dirichlet Allocation with the AP dataset, we (1) provide the first empirical evidence for the existence of continuous low-energy paths between local maxima, (2) demonstrate that these paths preserve semantic interpretability of topics, and (3) quantify the generalization performance of solutions along MEPs. Our findings reveal previously unknown geometric properties of the ELBO landscape and provide practical insights for initialization strategies in variational inference.