Scripts and dependencies inside skills

Two review tracks

A package may ask an agent to execute code. Review the instruction that triggers execution and the code’s actual behavior separately. Even a well-behaved model can run unsafe code when a legitimate-looking skill tells it to do so.

SkillJect’s research includes auxiliary scripts in its attack setup. The general review lesson is broader: the visible prose and the executable resource must agree about the task.

Make the execution path explicit

List executable entry points, arguments, working directories, subprocesses, dependencies, and outputs. Identify where the code discovers paths dynamically or inherits credentials from its environment. Review what is reachable, not only what the normal example happens to use.

Ask the maintainer for an explanation when an executable component is opaque or its source does not match the distributed artifact. Lack of an explanation is a review gap, not proof of malicious intent.

Dependency resolution is another trust decision

Record exact dependency versions and the source used to resolve them. A package reviewed today can behave differently if it installs changing dependencies at runtime. A lockfile helps identify a resolved set but does not establish that those packages are benign.

Keep dependency review and package review linked. Re-review when a dependency changes, an installation script changes, or a new registry or remote source is introduced.

Run with a bounded environment

Use an isolated workspace with synthetic inputs, no production tokens, and only the network destinations needed for the test. Observe file reads and writes, process execution, and outbound requests. Compare them with the declared task.

Filesystem and network restrictions should work together. A restricted directory can still contain valuable data, and a network restriction alone does not prevent local destructive changes.

Decide using evidence

• Can each permission be explained by the task?
• Are unrelated files and environment variables accessible?
• Does the code call remote services without a declared purpose?
• Do failure paths broaden behavior or silently retry elsewhere?
• Can the team reproduce the reviewed version and revoke access?

If evidence is incomplete, keep authority narrow. “It ran once without a visible problem” is a test observation, not a safety conclusion.

Worked example: a helper introduces a second installer

A skill generates charts using a bundled helper. Its setup step invokes another package manager, which runs an installation hook. The instruction file can be entirely accurate about chart generation while the dependency path adds unreviewed execution. Reviewing the helper’s main function is not enough to explain what happens during installation.

Work through the evidence

Map entry points, interpreters, package sources, resolution rules, hooks, subprocesses, and external services. Separate commands needed to inspect an artifact from commands that execute it. Record what is pinned and what is resolved at runtime. Inspect a disposable copy before running the workflow with synthetic inputs and restricted file and network access.

Decision criteria and a useful review record

Approve the smallest dependency state that has been reviewed for the intended task. A missing dependency should produce a bounded failure, not permission to fetch an arbitrary replacement. Keep installation behavior and normal execution behavior as separate review tracks. An unexplained setup action is a reason to hold the package even when the final chart looks correct.