THE ARCHIVE / EDITION 08
Datasets & tools
A small, annotated shelf of primary evaluation resources and maintainer documentation.
A resource shelf, not a leaderboard. We link to original projects. We have not run a common independent evaluation, and do not rank these resources as safer or better.
| Resource | Useful for | Keep in mind |
|---|---|---|
| MaliciousSkillBench ↗Catalog record S16 | Detection across different sources and structural families. | Curated benchmark; do not read its class balance as ecosystem prevalence. |
| MalSkillBench ↗Catalog record S24 | Joint instruction/code analysis with runtime verification. | A sandbox-verified sample need not represent all deployed environments. |
| SkillTrustBench ↗Catalog record S20 | Structured labels for normal, suspicious, and malicious skill cases. | Review release and label rules; CC BY-NC-SA 4.0 terms apply to the dataset. |
| Skill-Inject ↗Catalog record S04 | Skill-file injection tasks and legitimate task compliance. | Task-pair benchmark, not a marketplace census. |
| SkillSec-Eval ↗Catalog record S08 | Lifecycle-oriented assessment questions and evaluation. | Framework repository and host choices determine applicability. |
| Cisco Skill Scanner ↗Catalog record S19 | Maintainer-documented, best-effort multi-method skill inspection. | No clean scan guarantees safety. Optional model/cloud features need separate review. |
| SecureAI-Scan listing ↗Catalog record S34 | A published example of static findings with evidence tiers and traces. | A solution-directory listing is not independent testing or an endorsement. |
Before using a dataset
- Freeze the release and retain its license.
- Check sample labels and original sources.
- Look for duplicates and related artifact families.
- Choose a split that matches your deployment question.
- Report false alarms as well as malicious detection.
Before using a scanner
- Read scope and limitations.
- Record version, policy, model, and configuration.
- Know whether content leaves your machine.
- Validate on relevant benign and malicious cases.
- Preserve evidence for the final review decision.