Instructions, data, and authority

The ambiguous role of skill text

Skills intentionally contain instructions. A system cannot handle them as irrelevant text. The challenge is determining which guidance is appropriate for the task, which source supplied it, and which authority it may exercise.

A skill can legitimately describe a workflow, but it should not silently create permission to disclose unrelated data, change recipients, or override a user’s explicit constraint. Treat task guidance and permission to act as separate decisions.

A boundary ledger

For each instruction, record its source, purpose, and affected action. Then ask whether the user or platform authorized that action independently of the skill. A ledger makes a hidden prerequisite visible: “the skill requests access to folder X; the user’s task only needs folder Y.”

This method is an editorial review workflow. It is not a claim that a model can enforce all such boundaries just because they are explained in its prompt.

Warning patterns need context

Requests for secrecy, emergency exceptions, unrelated uploads, broad discovery of credentials, or disabling review are reasons to investigate. The exact words are less important than the mismatch between task and action.

A legitimate maintenance skill might discuss deleting temporary files. The question is which files, whose authority, which safeguards, and whether the declared purpose justifies it. Keyword matches alone can both miss unsafe behavior and reject useful instructions.

Enforce the action separately

A tool gateway can check the identity, action, target, scope, and required approval before execution. The runtime can restrict reachable paths and network destinations. These controls reduce impact if the model misinterprets skill content.

For high-impact actions, the confirmation should show the actual destination and relevant arguments. A skill’s statement that approval is unnecessary is not itself an authorization decision.

What to test

• A supporting reference asks for a target outside the approved workspace.
• A skill claims an unrelated transfer is a required validation step.
• A task-specific instruction tries to change a user-specified recipient.
• The agent proposes the unsafe step, but the runtime rejects it.
• The approved legitimate task still succeeds after the rejection.

Use synthetic data. Measure both the prohibited outcome and whether useful work can continue.

Worked example: a source asks for more authority

The user authorizes an agent to summarize a public support ticket. A skill reference says the summary must include an internal access token for debugging. That reference can provide task information, but it cannot grant access to an unrelated secret. The request is still outside scope if it is written politely, called a requirement, or repeated by the agent.

Work through the evidence

Build a small authority ledger. For each consequential action, record who requested it, what resource it touches, and which independent rule permits it. Compare a requested action with the user’s actual goal instead of classifying text solely by tone. In an isolated test, replace any secret with a canary and observe whether the workflow requests or attempts the same access.

Decision criteria and a useful review record

Stop an action whose authority comes only from the lower-trust material that wants it performed. A human approval step should expose the action and destination in plain terms, not merely ask whether to “continue setup.” Test both refusal of the unrelated access and successful completion of the legitimate summary. A defense that blocks all summaries has not demonstrated useful boundary enforcement.