parser.md

Parser (Interpreter) and AST

src/nsengine/interpreter.ts, src/nsengine/interpreter_steps/*, src/nsengine/ast.ts

The class is named Interpreter for historical reasons. It does not interpret — it is a recursive-descent parser producing an AST.

Two parsing surfaces

Statements are handled by private methods on Interpreter, tried in a fixed order by parseStatement():

parseBlock            {  }
parseIfStatement      if / else if / else
parseWhileStatement   while
parseDeclaration      let / const
parseForStatement     for (…;…;…) and for (x in y)
parseWhileStatement   (duplicated — harmless but dead)
parseBreakStatement   break [N];
parseContinueStatement continue;
parseFunctionDeclaration function f(…) {…}
parseReturnStatement  return [expr];
                      ── fallback ──
                      expression statement, then consume ';'

Expressions are handled by the interpreter-step chain. Each step is a class implementing one precedence level.

The interpreter-step chain

Every step extends InterpreterStep:

abstract class InterpreterStep {
    name: string;
    description: string;
    nextStep: InterpreterStep | null;   // the next-tighter-binding step
    interpreter: Interpreter;
    verboseMode: boolean;
    execute(params?: InterpreterStepParams): ASTNode | null | undefined;
}

The chain is built in the Interpreter constructor, tightest binding first, then exposed as interpreter.expressionSteps:

Step class	Handles	File
InterpretStringLiteral	'…' "…" `…`	interpret_stringliteral.ts
InterpretPrimitive	numbers, booleans, null, ( … )	interpret_primitive.ts
InterpretArrayLiteral	[ … ]	interpret_arrayliteral.ts
InterpretObjectLiteral	{ … } — object or set	interpret_objectliteral.ts
InterpretIdentifier	bare names	interpret_identifier.ts
InterpretMemberAccess	a.b, a?.b	interpret_member_access.ts
InterpretIndexer	a[b]	interpret_indexer.ts
InterpretFunctionCall	f(…)	interpret_functioncall.ts
InterpretPreUnary	-x, !x	interpret_preunary.ts
InterpretPostUnary	x++, x--	interpret_postunary.ts
InterpretPowRoot	**	interpret_powroot.ts
InterpretMulDiv	* / %	interpret_muldiv.ts
InterpretAddSub	+ -	interpret_addsub.ts
InterpretComparison	< > <= >=	interpret_comparison.ts
InterpretEquality	== !=	interpret_equality.ts
InterpretAndOr	&& || ^	interpret_andor.ts
InterpretFunctionDefinition	function f(…) {…}	interpret_function_definition.ts
InterpretBinarySpread	... / … ranges	interpret_binaryspread.ts
InterpretAssignment	= += -= *= /=	interpret_assignment.ts
InterpretTernary	? :	interpret_ternary.ts

parseExpression() enters the chain at ternary — the loosest level.

The standard binary-step shape

Almost every binary step is the same loop:

let lnode = this.nextStep?.execute(params);
while (!this.interpreter.isEOF() && this.interpreter.match("PLUS", "MINUS")) {
    const operator = this.interpreter.previous().value;
    const rnode = this.nextStep?.execute(childParams);
    lnode = {
        type: "BinaryOp", operator, left: lnode, right: rnode,
        isKnownAtCompileTime: lnode.isKnownAtCompileTime && rnode.isKnownAtCompileTime
    };
}
return lnode;

while gives left associativity. Four steps — comparison, equality, assignment and binarySpread — use if instead of while, so only one operator of that level is accepted per expression. a < b < c, a == b == c and a = b = c do not chain; the trailing operator is left in the stream and the statement parser fails on it. Parenthesise, or split into separate statements. addSub, mulDiv, powRoot, andOr, memberAccess and ternary all use while and chain normally.

Backwards-looking nodes

Postfix constructs (., [], ()) can follow any primary, including a literal: [1,2,3][0], f().x, a.b[0].c(). Rather than a single postfix loop, each postfix step accepts params.backwardsLookingNode — an already-parsed left operand — and the postfix steps mutually re-enter each other after building a node:

let lnode = params?.backwardsLookingNode || this.nextStep?.execute(params);
…build the node…
let m = this.interpreter.expressionSteps.memberAccess.execute({...params, backwardsLookingNode: lnode});
if (m) lnode = m;
let i = this.interpreter.expressionSteps.indexer.execute({...params, backwardsLookingNode: lnode});
if (i) lnode = i;

Every such site first checks peek().type === 'EOS' and returns early, because the mutual recursion has no other natural termination.

InterpreterStepParams

interface InterpreterStepParams {
    returnFunctionCalls?: boolean;     // this call site needs the call's value
    isInFunctionDefinition?: boolean;
    backwardsLookingNode?: ASTNode;    // see above
    executeInStatementMode?: boolean;  // do not fall through to the next step
}

returnFunctionCalls is threaded down from every context that consumes a value (operands, arguments, initialisers, conditions). It becomes FunctionCallNode.requireReturn, which the compiler uses to decide whether to emit OP_PUSH_RETURN64 after the call. A call in statement position discards its result and leaves the stack balanced.

executeInStatementMode is used by parseFunctionDeclaration() so that a non-function token returns null (letting parseStatement move on) rather than descending the whole expression chain.

Lookahead

patternLookahead(pattern) matches a token-type pattern against the upcoming stream without consuming. Pattern elements may be:

• a literal token type — "FOR"
• "*" — wildcard, matches anything
• "?TYPE" — optional, skipped if absent
• an array — matches if the token type is in the array; an array containing "?" is an optional set, one containing "*" is a wildcard

This drives the two ambiguous decisions in the grammar:

// for-in vs C-style for
["FOR", "LPAREN", ["?", "DECLARE_VARIABLE", "DECLARE_CONSTANT"], "IDENTIFIER", "IN"]

// object literal vs set literal
["STRINGLITERAL", "COLON"]

patternLookahead returns true when the token stream runs out mid-pattern, and the for (k, v in obj) branch it guards is entirely commented out.

Compile-time constant tracking

Every ASTNode carries an optional isKnownAtCompileTime. Leaves set it (Number, String, Boolean, Null → true; Identifier → false), and every composite step propagates the conjunction of its children.

The compiler reads this flag on ArrayLiteral, SetLiteral and ObjectLiteral nodes and, when set, calls ast.getCompileTimeValue(node) to build the real JS value during compilation, emitting a single OP_LOAD_PTR.

ast.ts also exports isKnownAtCompileTime() and compileTimeSolve() — an older constant-folding path that is no longer called from anywhere. Scalar constant folding (1 + 2 → 3) is therefore not performed; only collection literals benefit.

AST node types

src/nsengine/ast.ts

Node	Key fields
ProgramNode	statements[]
BlockNode	statements[], sameScope?
NumberNode	value, dtype: 'int' | 'float'
StringNode	value
BooleanNode	value, dtype: 'bool'
NullNode	—
StringBuilderNode	string (with placeholders), expressions[]
IdentifierNode	value
UnaryOpNode	operator, operand, postfix?
BinaryOpNode	operator, left, right
TernaryOpNode	condition, left, right
AssignmentNode	operator, left, right
ConditionNode	condition, body?, elseBody?
LoopNode	loopType: 'for'|'while'|'foreach', initializer?, condition?, increment?, iterable?, body?
BreakNode / ContinueNode	level
ReturnNode	value?
DeclarationNode	identifier, initializer?, dtype?, constant?
FunctionDeclarationNode	name, arguments[], body
FunctionCallNode	left, arguments[], requireReturn, targetLocation
MemberAccessNode	object, member, nullCoalescing?
IndexerNode	object, indices[]
ArrayLiteralNode	elements[], compileTimeValue?
SetLiteralNode	elements[], compileTimeValue?
ObjectLiteralNode	properties: {key, value}[], compileTimeValue?

All nodes carry an optional line used for error reporting, stamped by associateCurrentLine().

Helpers

• astToString(node) — ANSI-coloured tree dump, used by main.ts and exported from the package for debugging.
• getASTNodeCount(node) — node count, useful for complexity limits.
• getCompileTimeValue(node) — evaluates a constant subtree to a real JS value.