LEMCA: LLM-Guided Synthesis of Efficient Mode-Switching Control Architectures

Conference on Robot Learning (CoRL) 2026

Authors

Affiliations

Arjun Krishna

University of Pennsylvania

Vincent Pacelli

Amazon Robotics

Dinesh Jayaraman

University of Pennsylvania

Abstract

Physical control tasks in the natural world, such as driving or object manipulation, frequently exhibit dramatic variations in sensory and compute complexity over time. Correspondingly, a natural resource-efficient choice for robot control is to dynamically switch between control modes with varying resource allocations. However, such “mode-switching controllers” (MSCs) have historically required laborious, expert-driven design and synthesis for each new task. Driven by these design difficulties, modern robotic control architectures often fall back to a wasteful “monolithic” one-size-fits-all structure, where resource allocation is permanently anchored to the hardest, most resource-intensive task phases. To facilitate the design of performant yet efficient MSCs, we propose LLM-Guided synthesis of Efficient Mode-Switching Control Architectures (LEMCA). LEMCA represents MSC designs as interpretable programs to be iteratively refined in an evolutionary loop. To evaluate design fitness, we propose MSC-compatible extensions of automated controller synthesis approaches, such as reinforcement learning in simulation. LEMCA then leverages the semantic priors, reasoning, and coding capabilities of Large Language Models (LLMs) to iteratively edit controller modes, their corresponding sensory-compute resource allocations, and mode transitions. Our experiments across diverse control benchmarks show that LEMCA consistently discovers strategies that surpass the Pareto frontier of monolithic designs by reclaiming wasted resources during “easy” task phases. LEMCA thus presents an automated, low-effort path to synthesize resource-efficient MSC designs.

Links

arXiv OpenReview Code

Supplementary Material

The supplementary material is organized as follows, please click on the links below to navigate to the corresponding sections:

• Task Description
  Provides an overview of the following tasks LEMCA is evaluated on. The details of the task dynamics, sensor library, and associated costs are provided. See more at link.
  • point-to-disk
  • RF-DMC
    • cartpole-swingup
    • cup-catch
    • finger-spin
  • clutter-nav
• Best Designs
  This presents an interactive dashboard to via the best mode-switching controller designs synthesized by LEMCA for each task + performance target. The dashboard outlines the source code of the best design along with its performance metrics and rollouts. View the dashboard at link.

Citation

BibTeX citation:

@inproceedings{krishna2026lemca,
  title        = {LEMCA: LLM-Guided Synthesis of Efficient Mode-Switching Control Architectures},
  author       = {Krishna, Arjun and Pacelli, Vincent and Jayaraman, Dinesh},
  booktitle    = {Conference on Robot Learning (CoRL)},
  year         = {2026},
  note         = {To appear},
  eprint       = {2609.21319},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url          = {https://arxiv.org/abs/2609.21319}
}

For attribution, please cite this work as:

Arjun Krishna, Vincent Pacelli, and Dinesh Jayaraman. “LEMCA: LLM-Guided Synthesis of Efficient Mode-Switching Control Architectures.” In Conference on Robot Learning (CoRL), 2026.