October 9th/COLM 2026/Hilton Union Square, San Francisco

Workshop on Agent Behavior

Advancing the scientific study of how AI agents do and should behave.

Supported By

Anthropic logo
Google logo
OpenAI logo

The dominant paradigm in AI agent research is heavily capability-centric. Agents are primarily evaluated on what they achieve, and progress is measured by rigidly closing the gap between observed and desired performance. Though this has undoubtedly driven remarkable advances, it represents an incomplete account of agency that completely sidesteps a critical question: how do agents achieve what they achieve? A system that achieves a goal through opaque, brittle, or socially harmful processes raises deep concerns that traditional test metrics simply do not capture.

We propose a workshop centered on this complementary question, which we refer to as the study of agent behavior. This encompasses the full range of both observable and latent processes governing an agent's actions, including its decision-making strategies, its interaction patterns, its internal representations, and its responses to interventions.

Topics

1

Behavioral evaluations

How should we evaluate agent behavior beyond pure task success? We are exploring comprehensive behavioral test suites alongside causal or mechanistic explanations to understand exactly why agents behave as they do.

2

Agentic interactions

Covering rigorous studies of behavior generation, social simulation, and complex multi-agent dynamics where agentic behavior is inherently relational and interactive.

3

Interventions

Focusing on highly actionable methods like post-training, environment design, and structural constraints needed to reliably steer agents toward strictly desired behaviors.

4

Social and ethical implications

How do behavioral properties intersect with accountability? We investigate how to make this entire behavioral perspective deeply actionable for both policymakers and deployment practitioners.

5

Behavior foundation models

Developing the fundamental architectures and underlying models intrinsically designed to capture, generate, or simulate diverse AI behaviors.

However, we strongly welcome any submission that advances the understanding of AI agents through a behavioral lens, including bold work that bridges or directly challenges these established categories.

Accepted Papers

101 papers · Alphabetical by title

01

A Single-GPU Recipe for Specification-Grounded Industrial Decision Support

Zixiang Xu (University of Southern California), linna yu (Yuvous), miao xie (YUVOUS LLC)

02

ACCORD: Action-Conditioned Contextual Grounding for Language Agents

Lai Jiang (University of Illinois at Urbana-Champaign), Cheng Qian (University of Illinois at Urbana-Champaign), Zhenhailong Wang (University of Illinois Urbana-Champaign), Pan Lu (Stanford University), Heng Ji (University of Illinois, Urbana-Champaign), Hao Peng (University of Illinois Urbana-Champaign)

03

AgentAbstain: Do LLM Agents Know When Not to Act?

Xun Liu (University of Illinois at Urbana-Champaign), Yi Evie Zhang (University of Illinois at Urbana-Champaign), Vira Kasprova (University of Illinois at Urbana-Champaign), Parisa Rabbani (University of Illinois at Urbana-Champaign), Pardis Sadat Zahraei (University of Illinois at Urbana-Champaign), Tianyu Zhang (University of Illinois at Urbana-Champaign), Ali Ebrahimpour-Boroojeny (University of Illinois at Urbana-Champaign), Varun Chandrasekaran (University of Illinois Urbana-Champaign)

04

Agents Silently Drop Implied Obligations: Instruction-Conditional Shortcuts in Tool-Use Agents

Tony O'Halloran (Andromede AI)

05

AI Assistants Overassist

Verona Teo (Stanford University), Raghav Jain (University of California, San Diego), Tobias Gerstenberg (Stanford University), Max Kleiman-Weiner (Google; University of Washington; Common Sense Machines)

06

Align as Act: Innovations-Based Reward Decomposition for LLM Agents

Sojeong Rhee (Korea Advanced Institute of Science & Technology), Seohui Bae (LG AI Research), Jongeui Park (Korea Advanced Institute of Science and Technology), Whiyoung Jung (LG AI Research), Soonyoung Lee (LG AI Research), Woohyung Lim (LG AI Research), Youngchul Sung (Korea Advanced Institute of Science and Technology)

07

Are Large Reasoning Models Interruptible?

Tsung-Han Wu (University of California, Berkeley), Mihran Miroyan (University of California, Berkeley), David M. Chan (University of California, Irvine; University of California, Berkeley), Trevor Darrell (Electrical Engineering & Computer Science Department), Narges Norouzi (University of California, Berkeley), Joseph E. Gonzalez (University of California, Berkeley)

08

Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents

Sina Hajimiri (École de technologie supérieure, Université du Québec; ServiceNow Inc), Masih Aminbeidokhti (ServiceNow Inc; École de technologie supérieure, Université du Québec), Jose Dolz (École de technologie supérieure), Ismail Ben Ayed (École de technologie supérieure, Université du Québec), Issam H. Laradji (ServiceNow), Spandana Gella (ServiceNow Inc), Nicolas Gontier (Servicenow Research)

09

Auditing Agent Harness Safety

Chengzhi Liu (University of California, Santa Barbara), Yichen Guo (University of California, Santa Barbara; Huazhong University of Science and Technology), Yepeng Liu (University of California, Santa Barbara), Yuzhe YANG (University of California, Santa Barbara), Qianqi Yan (University of California, Santa Barbara), Xuandong Zhao (UC Berkeley), Wenyue Hua (Microsoft), Sheng Liu (Stanford University), Sharon Li (University of Wisconsin - Madison), Yuheng Bu (University of California, Santa Barbara), Xin Eric Wang (University of California, Santa Barbara; Simular)

10

Augmented Hypothesis Testing with Persona-Based LLM Simulations

Ziyad Benomar (Amazon), Aymen Al Marjani (Amazon), Paul Missault (Amazon), Saab Mansour (Amazon)

11

Behavior Beyond Text: Evaluating and Explaining Refusal Transfer Failure in LLM Agents

Aniekan-abasi Inyang (Clarkson University), Ekagra Gupta (Arizona State University), Adam Al Gharib (Carleton University), Raj Saha (Harvard University)

12

Behavioral Game Theory for Collaborative LLM Agents

Chengrui Qu (California Institute of Technology), Yizhou Zhang (California Institute of Technology), Nicolas Lanzetti (Deparment of Computing + Mathematical Sciences, California Institute of Technology), Eric Mazumdar (Deparment of Computing + Mathematical Sciences, California Institute of Technology)

Call for Contributions

We invite contributions that advance the scientific study of agent behavior across a wide range of topics and methodologies. We plan to announce acceptances by July 24th at the latest.

All submissions should be made via OpenReview.

Submission deadline

June 23, 2026 (AoE)

June 30, 2026 (AoE)

Submissions close in

Papers

We solicit non-archival papers (4–9 pages long) formatted in the standard COLM template.

Submissions undergo double-blind peer review. Preprints and concurrent submissions are explicitly encouraged.

Benchmarks

We seek proposals for new benchmarks to evaluate frontier AI agent behavior. Submissions should use this LaTeX template, and be 1-2 pages long. The template also contains more information about the expected format and content.

NOTE: this track is for benchmark proposals only (no implementation or results needed at this stage).

If selected, we will provide credits for running benchmarks, a harness for building the benchmarks, and support the creators in implementing an open-source version.

We invite creators of accepted benchmarks to collaborate towards a large-scale agent behavior evaluation suite and paper.

🏆 We plan to give one Best Paper Award along with several oral presentation slots.

Contact Us

For questions regarding the workshop, submissions, or participation, please contact the organizers at: