Execution Pipeline

Understand how a Soma script travels from source text to chart output — Lexer, Parser, compile cache, scope construction, and the tree-walk runtime.

Overview

Lexer

SomaLexer.ts

Converts raw source text into a flat list of typed tokens. Each token carries its line number for error reporting. Whitespace (spaces, tabs) and comments (# …) are consumed but never emitted to the parser. The lexer is line-oriented: keywords and identifiers are distinguished by two fixed sets — KEYWORDS and TYPES.

Token type	Content
KEYWORD	input output param let as for in if else while and or not true false break continue
TYPE	int float bool
IDENT	user variable and function names
NUMBER	integer or decimal literals
OP	+ − * / % ^ > < >= <= == != && \|\| & \| >> << ~ !
ASSIGN	=
COMPOUND_ASSIGN	+= −= *= /= %=
NULL_COALESCE	??
LBRACKET / RBRACKET	[ ]
LPAREN / RPAREN	( )
LBRACE / RBRACE	{ }
COMMA	,
QUESTION / COLON	? :

Parser → IR

SomaParser.ts

A single-pass recursive-descent parser. It first collects all declaration statements (input / output / param / let at the top level), then parses the body into a list of IRStmt nodes. The parser also computes a 32-bit integer hash of the source text using a fast djb2-style polynomial — used for compile caching.

Compile cache

SomaProc.ts

Before compiling, the node checks whether node.data._codeHash matches hashCode(somaCode). If it does, the cached node.data._compiled IR is reused and the parser is not invoked again. Any change to the script text invalidates the cache.

Scope construction

SomaProc.ts

Each declared input is resolved from the graph edge map. The upstream node's resultData.values is coerced to Float64Array via coerceInput(). Param values are merged from node.data.paramValues with the parsed defaults as fallback; int params are Math.trunc()'d. All inputs and params are placed in a SomaScope and the output slots are pre-populated with null.

Runtime (tree-walk)

SomaRuntime.ts

executeIR() walks the IRStmt array and evaluates each node recursively. Arithmetic and math functions work element-wise: when an operand is a Float64Array a new array is produced; scalar operands broadcast. Indicator functions (sma, ema, stdev) are loaded asynchronously once at startup via ensureIndicatorsLoaded() and reused on every execution. Outputs are collected from scope.outputs and written back to resultData as (number|null)[].

IR Reference

The Soma IR is a plain TypeScript discriminated union defined in SomaTypes.ts. All nodes carry a kind field that the runtime dispatches on. There are no opcode numbers, bytecodes, or foreign objects — the tree is inspectable as ordinary JSON.

IRExpr — expression nodes

kind	Fields	Description
lit	value: number	Numeric literal: 3, 1.5, etc.
array_lit	elements: IRExpr[]	Array literal: [1, 2, 3]. Each element is itself an IRExpr.
var	name: string, region: VarRegion	Variable read. region is one of: input \| output \| local \| param.
index	array: IRExpr, idx: IRExpr	Subscript access: arr[n]. Inside a for loop, idx 0 = current bar, 1 = previous bar.
call	fn: string, args: IRExpr[]	Built-in function call. fn is the function name; args is the argument list.
binop	op: BinOp, left: IRExpr, right: IRExpr	Binary operation. Element-wise when either operand evaluates to Float64Array.
unary	op: "-" \| "!" \| "~", operand: IRExpr	Unary negation, logical not, or bitwise not.
ternary	cond: IRExpr, then: IRExpr, else: IRExpr	Conditional: cond ? then : else.
null_coalesce	left: IRExpr, right: IRExpr	NaN-coalesce: a ?? b. Returns right when left evaluates to NaN.

IRStmt — statement nodes

kind	Key fields	Description
let	name, value, letType?, reAssign?	Scoped declaration in the current stack frame. letType is the optional type annotation.
local_assign	name, value	Re-assignment: walks the scope chain upward and updates the frame that owns the name.
assign	name, value	Output assignment. Writes to scope.outputs[name].
index_assign	name, idx, value	Indexed assignment: arr[i] = value. Works on both output buffers and local Float64Array variables.
for	indexVar \| null, valueVar, source, body	Iterates a Float64Array. indexVar may be null when the for-in form is used without an index.
if_stmt	cond, then, else	Conditional branch. else is an empty array when there is no else clause.
while_stmt	cond, body	Conditional loop. Executes body as long as cond is truthy.
break_stmt	—	Exits the innermost for or while loop via a thrown sentinel.
continue_stmt	—	Skips the rest of the current iteration via a thrown sentinel.
expr_stmt	expr	Standalone expression statement — the evaluated result is discarded.
print_table	labels, args	Expanded form of print(): labels hold the source-text of each argument for display in the console.

SomaScope

The runtime receives a single SomaScope object that serves as the root environment for the entire script execution. It is constructed fresh for every graph evaluation — there is no persistent mutable state between runs.

Field	Type	Description
inputs	Record<string, SomaValue>	Read-only during execution. Maps input channel names to their coerced Float64Array values.
outputs	Record<string, SomaValue>	Written by assign and index_assign statements. Initially null for each declared output.
frames	ScopeFrame[]	Scope chain. Index 0 is the outermost (script-level) frame. A new frame is pushed on entry to each block and popped on exit.
params	Record<string, number>	Read-only during execution. Contains the resolved (and type-coerced) parameter values.

Variable resolution

When the runtime reads a variable, it uses the region tag from the IR var node to choose the lookup table directly — there is no linear search through frames. The priority order is:

param — look in scope.params.
input — look in scope.inputs.
output — look in scope.outputs.
local — walk scope.frames from the innermost outward and return the first match.

Compile Cache

Parsing is O(n) in source length but is still noticeable for large scripts. The cache avoids re-parsing when the code has not changed. It is stored directly in the node's data object — no external cache store is required.

const hash = hashCode(data.somaCode);         // djb2-style 32-bit hash

if (data._codeHash !== hash || !data._compiled) {
  const result = compile(data.somaCode);       // Lexer + Parser
  if ('errors' in result) {
    // surface error to the node, abort
    return null;
  }
  data._compiled = result;                     // cache the IR
  data._codeHash = hash;                       // cache the hash
  // sync parsed declarations back to node.data
  data.inputs  = result.inputs;
  data.outputs = result.outputs;
  data.params  = result.params;
}

Graph re-evaluation: The graph is re-evaluated every time an upstream data source updates (e.g. new OHLCV bars). The compile cache ensures the parser runs only once per unique script text, even if the graph is evaluated thousands of times.

Input Coercion

Upstream nodes store their results in resultData.values as (number | null)[]. Soma needs Float64Array for efficient element-wise arithmetic. The coerceInput() helper bridges the gap:

function coerceInput(resultData: any): SomaValue {
  if (!resultData) return null;
  if (resultData.values && Array.isArray(resultData.values)) {
    return toF64(resultData.values);   // null → NaN, number → float64
  }
  return null;
}

toF64() converts a plain JS array to a Float64Array, replacing null with NaN. On the output side fromF64() performs the reverse, converting NaN back to null for downstream compatibility.

Error Handling

Errors flow through two distinct phases:

Compile-time errors

The parser returns { errors: SomaCompileError[] } when it encounters syntax problems. Each error has a line number and a message. They are surfaced on the node widget as a red badge. The previous valid IR remains cached so the graph can still render.

Runtime errors

The runtime wraps executeIR() in a try/catch. Uncaught JS exceptions (e.g. division by zero on a scalar, out-of-bounds access) are caught and written to node.data.error as a string prefixed with Runtime error:.

Constants Editor & Intellisense