When Agent Skills Break: Measuring Drift in Procedural-Knowledge Repositories

Agent Skills are portable SKILL.md packages of procedural knowledge that LLM coding and research agents load on demand. Controlled benchmarks such as SkillsBench ask whether a skill helps on a fixed task; operators also need observational answers about whether living skill repositories keep working as referenced files and endpoints change. We propose a five-axis measurement framework for skill observability and focus on skill drift: repository-unresolvable references inside skill packages. We report a purposive drift-positive case series of 51 independent public repositories (19,231 SKILL.md files; 2,491 deterministic in-repository broken-reference findings). The corpus characterises how drift appears and persists when present; it is not a prevalence sample. A stratified single-annotator audit finds precision 0.973 (143/147; Wilson 95% CI 0.932–0.989); recall remains unclaimed. Full-history replay classifies only 3 findings as regressed versus 2,488 born-broken; the median finding shipped broken for 27 days (maximum 214). A controlled behavioural experiment (3 skills, 2 agent runtimes, 360 trials) shows that repairing one broken reference raises reference-dependent accuracy from 0/3 to 3/3, that as-found drift can leave agents worse than a no-skill baseline, and that forbidding honest abstention can turn missing documentation into confident fabrication. We release an open CI/pre-merge detector that blocks the deterministic drift class. A post-sample re-check of the same pins shows many broken references still open at HEAD, while a star-biased scan of other popular catalogs finds most offline-clean, consistent with drift concentrating outside the most curated trees. The observational evidence complements controlled efficacy evaluation and motivates treating skill reference integrity as a continuous-integration concern.