Trace an agent's path before rewriting its prompt
Repeated experience with live systems suggests that most failures occur outside the . The first step is to record the start and end of every , save fixed details whenever material enters the , and count each time the path branches.
A large platform is unnecessary at first; simple timestamped records that remain available after the goes live can be enough. These records can reveal that the correct document was found but split before the needed date, a tool returned a data field with an unexpected name, or an agent succeeded four times before receiving a malformed fifth input absent from its test examples.
Reading the final answers or rewriting the prompt cannot expose or repair these failures. The useful loop is to trace the entire execution path and find the first point where it differs from the expected path.
Key points
- Record when every execution step starts and ends before editing the prompt.
- Save when material enters the and count every path branch.
- Check whether document splitting removed information needed by a later step.
- Confirm that tool output field names match the expected format.
- Trace the full path to find the first difference from the expected behavior.