Sn Tin

Learn Tin · Part 2: Agents · Chapter 18

Guarded tools and approval

A person approves every call to a dangerous tool.

Some tools shouldn't run because a model decided they should: deleting files, sending money, moving a robot arm. Mark them guarded, and a person approves every call.

A guarded tool

When the model asks for a guarded tool, the agent loop pauses and someone is asked. Here no one is there to answer, so the call is denied. The tool doesn't run, and the model is told to carry on without it:

denied.tintin run denied.tin
/// Permanently delete a file.
guarded tool fn deleteFile(path: string): string => "deleted {path}"

agent Janitor {
  model: "mock"
  tools: [deleteFile]
  on toolResult(name, output, ok) => print("  {name} -> {output}")
}

fn main() {
  print(Janitor("deleteFile 'old-backup.zip'"))
}
  deleteFile -> Tool call denied by the user. Do not retry it; continue without it.
mock: error: Tool call denied by the user. Do not retry it; continue without it.

With tin run --yes, every call is approved:

approved.tintin run approved.tin --yes
/// Permanently delete a file.
guarded tool fn deleteFile(path: string): string => "deleted {path}"

agent Janitor {
  model: "mock"
  tools: [deleteFile]
  on toolResult(name, output, ok) => print("  {name} -> {output}")
}

fn main() {
  print(Janitor("deleteFile 'old-backup.zip'"))
}
  deleteFile -> deleted old-backup.zip
mock: deleted old-backup.zip
Where the program runsWho is asked
tin run in a terminalyou, with a y/N prompt showing the tool and its arguments
tin run --yesnobody: everything is approved
no terminal (a pipe, CI)nobody: everything is denied
the playgroundyou, in a browser dialog

The compiler checks there's no way around it

A prompt at run time would be worth little if other code could just call the function. So the compiler proves it can't. Calling a guarded tool directly is an error:

direct.tintin check direct.tin
/// Permanently delete a file.
guarded tool fn deleteFile(path: string): string => "deleted {path}"

fn main() {
  print(deleteFile("old-backup.zip"))     // skipping the person is not allowed
}
error: 'deleteFile' is a guarded tool, and this call skips approval
 --> direct.tin:5:9
  |
5 |   print(deleteFile("old-backup.zip"))     // skipping the person is not allowed
  |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  = note: path: main → deleteFile
  = note: give the tool to an agent (the agent loop asks before it runs), or call it from another guarded tool fn

And it follows calls all the way down. Here an ordinary tool reaches a guarded function two calls away. The error shows the whole path:

sneaky.tintin check sneaky.tin
/// Move money. Cannot be undone.
guarded fn wire(to: string, cents: int): string => "sent {cents} to {to}"

fn payInvoice(id: int): string => wire("vendor-{id}", 100)

/// Pay an invoice by id.
tool fn pay(id: int): string => payInvoice(id)     // an unguarded tool, two calls away from wire

agent Clerk { model: "mock", tools: [pay] }

fn main() {
  print(Clerk("pay 7"))
}
error: tool 'pay' can run 'wire', which is guarded, without anyone approving it
 --> sneaky.tin:4:35
  |
4 | fn payInvoice(id: int): string => wire("vendor-{id}", 100)
  |                                   ^^^^^^^^^^^^^^^^^^^^^^^^
  = note: path: pay → payInvoice → wire
  = note: mark it `guarded tool fn pay` so the agent loop asks first
  = fix: mark pay as guarded

The note is also a fix: tin fix marks pay as guarded for you. This check is exact, not a guess, because tool parameters are plain data and Tin has no global variables: the only code a tool can run is code its body names.

Asking yourself: approve

Your own code can call a guarded tool too, through approve. It shows a person the exact call and returns a Result: Ok with the tool's result on a yes, and Err on a no.

approve.tintin run approve.tin
/// Permanently delete a file.
guarded tool fn deleteFile(path: string): string => "deleted {path}"

fn main() {
  // approve shows a person the exact call, and runs it only on a yes
  match approve deleteFile("draft.md") {
    Ok(msg) => print(msg)
    Err(why) => print("not done: {why}")
  }
}
not done: denied: deleteFile
approve-yes.tintin run approve-yes.tin --yes
/// Permanently delete a file.
guarded tool fn deleteFile(path: string): string => "deleted {path}"

fn main() {
  // approve shows a person the exact call, and runs it only on a yes
  match approve deleteFile("draft.md") {
    Ok(msg) => print(msg)
    Err(why) => print("not done: {why}")
  }
}
deleted draft.md

Tasks

●○○ Task 1: An admin agent

Give an agent a listUsers tool anyone can use and a removeUser tool that needs approval. Print each call, and run it with --yes.

Solution
task-1.solution.tintin run task-1.solution.tin --yes
/// List the users.
tool fn listUsers(): string[] => ["ada", "lin", "old-test-user"]

/// Remove a user account. Cannot be undone.
guarded tool fn removeUser(name: string): string => "removed {name}"

agent Admin {
  model: "mock"
  tools: [listUsers, removeUser]
  on toolCall(name, input) => print("  call {name} {input}")
}

fn main() {
  print(Admin("use listUsers"))
  print(Admin("removeUser 'old-test-user'"))
}
  call listUsers {}
mock: ["ada","lin","old-test-user"]
  call removeUser {"name":"old-test-user"}
mock: removed old-test-user

●●○ Task 2: Make it compile

Fix the payment program from above the right way: keep wire guarded, and make a person approve every payment.

Solution
task-2.solution.tintin run task-2.solution.tin
/// Move money. Cannot be undone.
guarded fn wire(to: string, cents: int): string => "sent {cents} to {to}"

fn payInvoice(id: int): string => wire("vendor-{id}", 100)

/// Pay an invoice by id.
guarded tool fn pay(id: int): string => payInvoice(id)    // now a person approves every payment

agent Clerk { model: "mock", tools: [pay] }

fn main() {
  print(Clerk("pay 7"))
}
mock: error: Tool call denied by the user. Do not retry it; continue without it.

Marking pay as guarded tool fn is what the compiler suggested. Removing guarded from wire would also compile, but then nobody would be asked.

What's next

Next: Capabilities. The compiler proves what each agent can reach.

In the reference: Guarded tools and approval · The compiler checks the approval path · approve