Reference
How it works
From source text to a WebAssembly module: the compiler pipeline, the memory layout and the runtime.
The pipeline
The compiler is TypeScript with no dependencies, so it runs anywhere JavaScript does, including
this site's playground. Node runs the .ts files directly; there is no build step.
| Stage | Does | Lines |
|---|---|---|
src/lexer.ts | source text → tokens | 270 |
src/parser.ts | tokens → syntax tree | 968 |
src/checker.ts | syntax tree → typed IR | 3745 |
src/caps.ts | typed IR → proof or error | 322 |
src/codegen.ts | typed IR → WebAssembly bytes | 435 |
src/prelude.ts | the standard library, written in Tin | 1079 |
src/runtime.ts | the host: event loop glue and the agent loop | 1363 |
lexer.ts. Newlines are not tokens: each token remembers whether a line break came before it, which is how Tin does without semicolons. String literals are split at {…} so interpolations are parsed as real expressions. Doc comments are kept and attached to the next token.
parser.ts. Hand-written recursive descent with precedence climbing for operators. x |> f(y) is rewritten to f(x, y) here, so later stages never see a pipeline.
checker.ts. Type inference and checking, and all lowering. Described below.
caps.ts. Walks what every function can reach and rejects any path from unapproved code to guarded code.
codegen.ts. Writes the binary format directly. No wabt, binaryen or LLVM.
What the checker does
The checker turns the syntax tree into a small typed IR in which every construct a WebAssembly machine does not have is already gone.
- Generics are monomorphised. A generic function is checked and compiled once
per distinct set of type arguments, on demand.
map<int, string>andmap<float, float>are two separate WebAssembly functions with no boxing. Only functions reachable from your code are instantiated, which is also how the unused parts of the standard library are left out. - Inference is local and bidirectional. Each expression is checked with an
optional expected type. That is how
[],None, lambda parameters and typed agent replies get their types, and why Tin never needs type arguments at a call. - Closures are converted. A lambda becomes a top-level function taking an environment pointer, plus a heap record holding the function's table index and the captured values. A lambda that captures nothing is a static constant.
matchbecomes branches on the variant's tag, after the exhaustiveness check.- Value semantics become copies. Where an array or struct enters or leaves a variable that can change, the checker inserts a deep copy.
- Checked arithmetic is inlined. Integer
+and-test the sign bits of the operands and the result;*compares against the exact product computed in floating point. spawnbecomes a thunk. The call's arguments are evaluated, stored in a record with a result slot, and the runtime is asked to start the thunk on a new stack.
Code generation
The IR maps almost one-to-one onto WebAssembly instructions, so the generator is a single
recursive function. Tin's int, bool and every pointer are
i32; float and Duration are f64. A module contains:
- imports from the host, only those the program uses;
- the allocator and four small trampolines the host uses to call closures;
- your functions and the library functions you reached;
- a function table for closures and indirect calls;
- one data segment holding string literals and static values;
- a
namesection, so traps show Tin function names; - a
tin.metasection describing tools, agents and type layouts.
Memory layout
Everything on the heap lives in the module's linear memory.
| Value | In memory |
|---|---|
string | a pointer to [length: i32][UTF-8 bytes] |
T[] | a pointer to a header [length][capacity][data pointer]; items are 4 or 8 bytes each (a bool item is 1 byte); the data block doubles when full |
| struct | a pointer to its fields in declaration order, 4 or 8 bytes each |
| sum type | a pointer to [tag: i32][fields of that variant]; variants without values are shared constants |
| function value | a pointer to [table index][captured values] |
Task<T> | a pointer to [done flag][result] |
Agent | a handle, an index into the runtime's table of agents |
The allocator keeps a free list per size class and grows the memory when it has to. There is
no garbage collector. Because a var's storage is never shared, the compiler knows
exactly when it dies, and frees what a var owns when its block ends, when it is
reassigned, and when an array or struct held in one of its fields or items is replaced. A loop that
rebuilds a value on every pass therefore runs in constant memory.
Call arguments are borrowed, not copied: a function copies only what it keeps or returns, so a
call result is a new value its caller owns, and a let bound to one is freed at the end of
its block too.
Strings are owned like arrays: copied when stored in a var, a struct or an array, and
freed with their owner. The temporary pieces of an interpolated string, a message built only to be
printed, and a string only measured or searched are freed straight away.
Leaving early with return, break, continue or ?
frees what the blocks being left own. Copies of arrays of plain values are one block copy.
Closures are immutable, so copies share one environment with a reference count; the last
reference frees it and what it captured. A spawned call frees its argument copies when it finishes.
What remains is the runtime's small per-task record.
Running out of memory is a panic, out of memory, not a crash.
The runtime
A Tin module imports 60 host functions at most, all from one module named
tin. The ones marked suspends may pause the calling task.
| Import | Signature | |
|---|---|---|
print | (string) | |
write | (string) | |
panic | (string) | |
float_to_str | (float) → string | |
float_fixed | (float, int) → string | |
parse_float | (string) → float | |
parse_ok | () → bool | |
sin | (float) → float | |
cos | (float) → float | |
tan | (float) → float | |
exp | (float) → float | |
ln | (float) → float | |
pow | (float, float) → float | |
random | () → float | |
now | () → duration | |
sleep | (duration) | suspends |
duration_to_str | (duration) → string | |
spawn | (int, int) | |
await_task | (int) | suspends |
task_done | (int) | |
task_stop | (int) | |
extern_call | (int, int) → int | suspends |
js_call | (int, int) → int | suspends |
fuel | () | |
task_wait | (int, duration) → bool | suspends |
fetch | (string) → string | suspends |
fetch_ok | () → bool | |
input | (string) → string | suspends |
approve_call | (string, string) → bool | suspends |
env | (string) → string | |
args | () → string | |
exit | (int) | |
agent_new | (int, string, string, int, int, int, int, int, int) → agent | |
agent_ask | (agent, string, int) → int | suspends |
agent_try | (agent, string, int, int) → int | suspends |
agent_reset | (agent) | |
agent_fresh | (agent) → agent | |
agent_turns | (agent) → int | |
agent_tokens | (agent) → int | |
agent_save | (agent) → string | |
agent_restore | (agent, string, int) → int | |
voice_new | (int, string, string, string, duration, int, int, int, int, int, int, int, int) → agent | |
voice_open | (agent) → int | suspends |
voice_talk | (agent, int, int) → string | suspends |
voice_say | (int, string) | suspends |
voice_send | (int, int) | suspends |
voice_next | (int, int) → int | suspends |
voice_interrupt | (int) | suspends |
voice_stop | (int) | suspends |
voice_save | (int) → string | |
audio_mic | () → int | |
audio_read | (int) → int | suspends |
audio_speaker | () → int | |
audio_play | (int, int) | suspends |
audio_clear | (int) | |
audio_seconds | (int) → float | |
audio_rate | (int) → int | |
audio_tone | (float, duration) → int | |
audio_silence | (duration) → int | |
audio_level | (int) → float |
The event loop: JSPI
Tin has no event loop of its own. It uses the host's, through WebAssembly JavaScript Promise
Integration. When Tin calls a suspending import such as sleep, the engine parks the
whole WebAssembly stack and returns to the host's event loop; when the promise resolves, the stack
is resumed where it stopped. So a Tin function that sleeps looks like any other function: no
async, no state machine generated by the compiler, no function colouring.
Each task runs on its own stack. spawn, each tool call made by a model, and each
event handler is a separate entry into the module, so they interleave wherever one of them
suspends. The module keeps one global naming the running task; code saves it on its own stack
before a suspending call and restores it on resuming, which is how the runtime knows who spawned
what. To stop a task, the runtime rejects the promise it is waiting on, and the exception unwinds
that task's stack. Without JSPI (older engines), programs that never suspend still run; the suspending
functions report a clear error.
The agent boundary
The agent loop lives in the runtime, and tools live in the module. Values cross between them as
JSON on the model's side and as Tin values in memory on the module's side. The compiler makes that
possible by describing every type that crosses in the tin.meta section: for a struct,
its fields with their offsets; for a sum type, its variants with their tags. From those
descriptions the runtime:
- derives the JSON Schema it shows the model;
- validates the model's arguments and builds them in memory before calling the tool's export;
- reads the tool's result out of memory and turns it into JSON;
- does the same in the other direction for typed replies.
So the schema, the validation and the memory layout all come from one source: the type as the compiler saw it.
Code as action uses the same boundary twice. The model's program
is compiled to a second module whose only imports beyond the standard ones are
extern fn declarations of the agent's tools. A call packs its arguments into a record;
the runtime reads that record out of the sandbox's memory as JSON, runs the real tool in the main
module, and writes the result back into the sandbox. Fuel is a host call at the top of every
function and loop body, compiled in only for such programs.
Sizes and speed
Measured while building this page, on the machine that built it.
| What | Result |
|---|---|
Hello world, as a .wasm file | 701 bytes |
examples/agents/01-tools.tin, as a .wasm file | 3,180 bytes |
| Compiling that example, including the standard library | 2.1 ms |
| The compiler and runtime, bundled for the playground | about 110 kB before compression |
Compiled code runs as optimised WebAssembly. On one test, fib(35) with overflow
checks on took about 17 ms where the same function in plain JavaScript took 46 ms. That is one
micro-benchmark on one machine, not a claim about programs in general.
Known limits
- Memory is reclaimed only for what a
varowns, and value copies are eager. Both are the same planned piece of work: reference counting with copy-on-write. - A syntax error stops the run where it is; type errors are all reported (tin check).
- A program is a single file.
- WebAssembly's 32-bit address space caps a program at 4 GiB.
- The only host is JavaScript. The module does not use WASI.
- Stack depth is the engine's. Deep recursion ends in a reported stack overflow, not a crash.