Fixing why Hermes' self-generated skills don't actually work well
Skills that the generates for itself mid-conversation often don't work well in practice. The core problem is that these skills get saved as notes and then activated immediately without any checking. The proposed fix stops an agent from turning a successful conversation straight into an active skill, and instead introduces a lifecycle.
A skill moves through: Candidate (generated but not yet available to automatic routing) → Audited (its claims are extracted and checked) → Verified (evidence and tests have passed) → Active (eligible to be loaded automatically). At any point a skill can be sent to Rejected if it turns out inaccurate or not genuinely reusable. Even an Active skill can later become Stale over time, after which it gets Reverified or — kept around for inspection but no longer allowed to influence Hermes' behavior.
This isn't something that can be toggled today; it requires a change to Hermes' core. On top of the lifecycle, each skill would carry a manifest file recording its status, source type and URLs, the source revision, when it was verified, when that expires, which Hermes and Python versions it was verified against, each individual claim with its supporting evidence and test result, and the scope it applies to (, ).
Key points
- Hermes' self-generated mid-conversation skills were being saved as notes and activated immediately, causing them not to work well
- Proposed fix adds a lifecycle: Candidate → Audited → Verified → Active, with a Rejected branch for anything inaccurate or non-reusable
- Even Active skills can go Stale over time and then get reverified or
- Each skill would ship with a manifest recording status, source, date, evidence, and test results
- This requires a core change to Hermes — it isn't an existing option