The ever insightful Paul Taylor writing on ‘Free Agents’ in the LRB.
As LLMs have become more powerful and their responses less error-prone, the number of applications built this way has boomed. It is often useful to think of the prompt sent to an LLM as composed of different elements, some of which supply contextual information and others more detailed instructions. By specifying a set of different contexts to be used in different prompts, you can configure a set of distinct software agents to tackle different aspects of a problem – one could be the project manager, another the programmer and a third could run checks and find errors. Programmers refer to these characteristics as forming an agent’s ‘persona’. In the early days of LLMs, agentic programming rarely worked for long. Once you set the agents talking to one another the original goal would quickly get lost. The models were designed to converse with people and to include pleasantries in their responses, which, if consistently recycled back into the conversation, come to dominate the exchanges. Agents drifted away from their assigned tasks in a phenomenon known as ‘sycophancy collapse’.
Sycophancy collapse, is a new one on me. Brings to mind my thoughts on much medical education (or more accurately, medical educationalists).