A test to map how one concept moves through an LLM
This is an experiment to find how a handles one concept inside the model. The goal is not to point to one neuron, one hidden state, or one layer, but to recover a repeated that appears across layers.
The test asks many comparable questions about entities such as India, France, Japan, and Germany, including questions about capitals, currencies, animals, population, and languages. It then captures , attention, and , measures how selective individual neurons are, and builds activation-based graphs across .
Hundreds of contrast prompts are combined into shared s for each entity, and the overlap between entities is compared. The early finding is that the useful unit is not a single neuron or one activation vector, but a pattern spread across many neurons and layers.
Key points
- The experiment looks for repeated s inside a transformer.
- It compares many similar questions across entities such as countries.
- It records , attention, and across layers.
- The early signal is that concept handling is distributed across many neurons and layers.
- There is no direct agent-building or cost-saving method yet.