Files
plainleaf/client/space_lua/stdlib/table.ts
T
75cc79800f Space Lua: Align numeric/table semantics with Lua, align number formating, optimize loops allocations (#1823)
* Don't know what's going on with the Deno docker image, disabling the deno.lock file for now

* Another try

* Space Lua: Align numeric and table semantics with Lua

This change improves Space Lua compatibility with standard Lua 5.4,
focusing on numeric subtypes and table behavior. The test suite is
extended to lock in the expected semantics and should pass under both
Space Lua and a Lua interpreter.

SUMMARY OF CHANGES
------------------

Tighten Lua compatibility across evaluator and runtime:

- correct metamethod dispatch (`__index`, `__newindex`, `__call`,
  comparison metamethods),

- loop limits,

- raw metamethod lookups.

Rework numeric semantics to preserve Lua **integer** vs **float**
behavior:

- $0$ vs $0.0$ and $-0.0$,

- explicit zero kind representation, and

- updates arithmetic/bitwise coercions accordingly.

Improve parser correctness by rejecting **unary plus** with aligned Lua
errors and better parsing errors reporting.

Fix `stdlib` behavior to match that of Lua:

- `table` function `concat`, `insert`, `remove`, `sort` and `unpack`
  gain metamethod awareness and enforce Lua errors,

- `ipairs` iteration updated to stop on first nil and honor `__index`,

- `tonumber` updated to Luae conversion using `luaToNumberDetailed`,

- `math.modf` return corrected,

- `math.type` accuracy improvements for float/integer and $-0.0$,

- `math.pi` added.

Fix numeric subtypes for `/` and `^` operators so `math.type` matches
Lua results using tagging as well as unary `-`.

Expand test coverage:

- new `metamethods_test.lua` for Lua metamethod/operator semantics,

- extend arithmetic and length tests for zero-kind propagation and
  `rawlen` vs `__len` metamethod,

- Extend `math` test suite to test proper Lua alignment (`math.type` and
  more), and

- update context error expectations for Lua error messages.

RATIONALE
---------

Lua differs from JavaScript by having two numeric subtypes: **integer**
and **float**. Operators depend on the subtype: `+`, `-`, `*`, `//` and
`%` use integer mode when both operands are integers and float mode
otherwise. Bitwise operators require integers and `math.type(x)` reports
"integer" or "float". JavaScript has one numeric primitive type
(`number`) so a plain number value cannot record whether Lua considers
a value to be a float when the value has no fractional part (for example
$2.0$).

Lua also differs from "everything is IEEE 754 double" because the rules
are defined in terms of integer and float subtypes. Float operations
preserve IEEE 754 behavior including `NaN`, infinities and signed zero
($-0.0$) which affects results like $1/0.0$ versus $1/-0.0$. Integer
arithmetic does not preserve $-0$ and collapses it to $0$. Lua integer
arithmetic is exact within its integer range while JavaScript `number`
cannot exactly represent all integers in that range.

Lua numbers are integers or floats. Numeric strings coerce to integer or
float based on _lexical_ form. Each arithmetic operator selects the
result subtype from the operator rules and operand subtypes.  Integer
only operators (bitwise and `//` as integer division) require integer
representability. Mixed arithmetic promotes to float as needed.  Two
operators are **always float** typed: division (`/`) and exponentiation
(`^`) produce floats even if both operands are integers and even if the
numeric value has no fractional part. `math.type` reports that internal
subtype.

Tables are associative arrays and assigning `nil` removes a key. The
length operator `#` uses `__len` metamethod if present otherwise it uses
the raw length rule. `rawlen(table)` ignores `__len`. Without `__len`
Lua defines `#` as some boundary `N` such that `table[N]` is not `nil`
and `table[N+1]` is `nil`. If the table has holes (missing or `nil`
entries in the positive integer key sequence) the boundary may be non
unique so `#` is stable only for proper sequences without holes.

PERFORMANCE NOTES
-----------------

Numeric changes add small checks to preserve Lua integer and float
subtype semantics and avoid allocations except when the subtype would
otherwise be lost.

Some table operations may be slower due to stricter Lua 5.4 behavior
especially around length and sequence boundary handling which currently
requires extra metadata tracking and scans and cannot be avoided without
a completely different internal table representation.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Another try

* Replace `LuaFloatTag` plain objects with boxed Number for float tagging

* Restore pre-merge eval/numeric architecture and fix regressions

This commit restores the original branch architecture.

On top of the restored foundation, float-typed integer results (e.g.
`1.0 + 1.0` = `2.0`, `0.0 // 1.0` = `0.0`) are now correctly tagged via
`makeLuaFloat` so that `tostring` and `math.type` report them as floats.
The *unary minus* fast path for float literals and the `tonumber`
function also preserve float tagging.

Performance regressions from the merge are addressed by avoiding
`Number` boxing for non-integer floats (`3.14` needs no tag — it is
unambiguously float), adding *string key* fast paths in `LuaTable`
`has`/`rawGet`/`rawSet` to skip numeric normalization for the dominant
case, and inlining a `typeof` check in math standard library functions
to avoid function call overhead on plain numbers.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Replace boxed `Number` float tagging with plain tagged objects

* Replace `new Number()` boxing with plain tagged float objects for Lua
  float type tracking.

* Integer-valued floats that need type disambiguation are now
  represented as `{ value: number, isFloat: true }` instead of boxed
  `Number` objects with a symbol property.

* Pre-allocated singletons are used for positive and negative float
  zeros to avoid allocation entirely in common cases.

* Updated all detection, unwrapping, and coercion paths across
  `numeric.ts`, `runtime.ts`, `eval.ts`, `stdlib.ts`, and `stdlib/`
  modules to use the new `isTaggedFloat` type guard.

* Removed all `instanceof Number` checks.

* Deleted the `FloatKind` symbol and eliminated redundant helpers
  `isLuaFloat`, `isFloatTag`, `getZeroBoxKind` and `toPlainNumber` that
  became dead code.

* Simplified `math.type`, `luaToString`, `luaEquals`, `luaTypeName` and
  various other key normalization paths.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Add fast paths in `coerceNumericPair` for tagged float operands

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Fix copy/paste typo

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Avoid extra `LuaEnv` allocations in "For" and "ForIn" loops

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* [Space Lua] Reuse loop variable environment in `for` and `for-in` loops

Numeric `for` and generic `for-in` loops allocated a fresh `LuaEnv` on
every iteration to hold loop variables. But this is only necessary when
a closure inside the loop body captures the loop variable.

The optimization uses a two-level check computed at parse time. If no
function definition exists in the loop's subtree, environment reuse is
safe. When a function definition is present a deeper analysis walks the
block to determine whether any function body references the loop
variable names without them being shadowed by its own parameters. When
a closure captures a loop variable the loop fall back to per-iteration
allocation.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* [Space Lua] Format numbers using standard Lua rules

Standard Lua formats floats via C `sprintf("%.14g")` (14 significant
digits, scientific notation when shorter, exponent padded to 2 digits,
and a guaranteed `.0` suffix for integer-valued floats).

* Replace the old `luaFormatNumber` with JS `toPrecision(14)`-based
  implementation that reproduces this behavior.

* Integrates it so that `${}` expressions in the UI also display
  numbers correctly.

* Fixes tagged floats (`{ value, isFloat }`) were being stripped by
  `luaValueToJS` or matched as plain objects before reaching the number
  formatter. This caused `${}` expressions to render raw JS numbers.

Examples:

```
- ${tostring(2^63)}
- ${2^63}
- ${(2^63)}
```

All of the the above examples show correct `9.2233720368548e+18` now.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* [Space Lua] Fix `string.format` for floats and tagged numbers

Unwrap tagged floats before `printf`, handle `inf`/`-inf`/`-nan` in
`formatDouble`, and fix `%g` producing `0e+00` for zero. Hopefuly it's
enough to gain Lua formatting.

Tests:

```
- ${string.format("%.14g", 0.0)} - `0`
- ${string.format("%.14g", 1.0)} - `1`
- ${string.format("%.14g", 1/3)} - `0.33333333333333`
- ${string.format("%.14g", math.pi)} - `3.1415926535898`
- ${string.format("%.14g", 1e-10)} - `1e-10`
- ${string.format("%.14g", 1e18)} - `1e+18`
- ${string.format("%.14g", 2^63)} - `9.2233720368548e+18`
- ${string.format("%.14g", 2^53)} - `9.007199254741e+15`
- ${string.format("%.14g", 1.7976931348623e+308)} - `1.7976931348623e+308`
- ${string.format("%.14g", 5e-324)} - `4.9406564584125e-324`
- ${string.format("%.14g", 0/0)} - `-nan`
- ${string.format("%.14g", 1/0)} - `inf`
- ${string.format("%.14g", -1/0)} - `-inf`
```

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* [Space Lua] Add and integrate new `luaFormat` utility and test suite

* Add `luaFormat` string formatting function compatible with Lua, and
  integrate it across the codebase as a replacement for prior formatting
  approaches.

* Add extensive test suite.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Fix check

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* [Space Lua] Add `%a`/`%A` and `%q` format specifiers to `string.format`

Implement hexadecimal floating-point (`%a`/`%A`) and quoted literal
(`%q`) specifiers.

* Add `%a`/`%A` as IEEE 754 double decomposition with full flag, width
  and precision support.

* Add `%q` as producind valid Lua literals for strings, numbers,
  booleans and nil.

* Use `Math.PI` for `math.pi` to preserve full double precision.

* Remove Deno based test suite and replace it with native Lua test
  suite.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* [Space Lua] Add `%p` format specifier to `string.format`

In standard Lua, `%p` formats the internal C heap address of a value,
producing output like `0x55a3bc4e2f10`. It works on tables, functions,
threads, strings, and userdata (GC-ed objects). For `nil`, booleans, and
numbers it returns `(null)`.

In Space Lua, there are no *raw memory addresses* since the runtime is
JavaScript. Instead, `%p` assigns a *stable sequential integer* to each
object via a `WeakMap`, formatted as a 14-digit zero-padded hex value.
The key difference is that identifiers are deterministic and sequential
rather than random-looking heap addresses:

```lua
local t = {}

print(string.format("identifier: %p", t)) -- 0x00000000000001
print(string.format("the same:   %p", t)) -- 0x00000000000001
print(string.format("another:    %p", {}) -- 0x00000000000002
```

For strings, a regular `Map` is used so identical string content always
produces the same identifier.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Revert merge changes to the deno.json

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Replace `interface` with `type`

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Remove `has_math()` relict function test

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Refactor loop to map

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Remove `Deno.remove("deno.lock")` weirdness

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Refactor: Early return undefined in `astNumberKind` instead of assigning

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

---------

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
Co-authored-by: Zef Hemel <zef@zef.me>
2026-02-12 15:22:12 +01:00

402 lines
11 KiB
TypeScript

import {
getMetatable,
type ILuaFunction,
LuaBuiltinFunction,
luaCall,
type LuaEnv,
luaEquals,
luaGet,
LuaMultiRes,
LuaRuntimeError,
luaSet,
LuaTable,
type LuaValue,
luaValueToJS,
singleResult,
} from "../runtime.ts";
import { asyncQuickSort, evalPromiseValues } from "../util.ts";
import { isTaggedFloat } from "../numeric.ts";
// For `LuaTable` honor `__len` when present; otherwise use raw array
// length. For JS arrays use `.length`.
function luaLenForTableLib(
sf: any,
tbl: LuaTable | any[],
): number | Promise<number> {
if (Array.isArray(tbl)) {
return tbl.length;
}
if (!(tbl instanceof LuaTable)) {
return 0;
}
const mt = getMetatable(tbl, sf);
const mm = mt ? mt.rawGet("__len") : null;
if (!(mm === undefined || mm === null)) {
const r = luaCall(mm, [tbl], sf.astCtx ?? {}, sf);
if (r instanceof Promise) {
return r.then((v: any) => Number(singleResult(v)));
}
return Number(singleResult(r));
}
return tbl.length;
}
async function luaLenForTableLibAsync(sf: any, tbl: LuaTable | any[]) {
const r = luaLenForTableLib(sf, tbl);
return r instanceof Promise ? await r : r;
}
export const tableApi = new LuaTable({
/**
* Concatenates the elements of a table into a string, using a separator.
* @param tbl - The table to concatenate.
* @param sep - The separator to use between elements.
* @param i - The start index.
* @param j - The end index.
* @returns The concatenated string.
*/
concat: new LuaBuiltinFunction(
async (sf, tbl: LuaTable | any[], sep?: string, i?: number, j?: number) => {
sep = sep ?? "";
i = i ?? 1;
if (j === undefined || j === null) {
j = await luaLenForTableLibAsync(sf, tbl);
}
const luaConcatElemToString = (v: any, idx: number): string => {
// Concat errors on nil and non-string or non-number values.
if (v === null || v === undefined) {
throw new LuaRuntimeError(
`invalid value (nil) at index ${idx} in table for 'concat'`,
sf,
);
}
if (typeof v === "string") {
return v;
}
if (typeof v === "number") {
return String(v);
}
if (isTaggedFloat(v)) {
return String(v.value);
}
const ty = typeof v === "object" && v instanceof LuaTable
? "table"
: typeof v;
throw new LuaRuntimeError(
`invalid value (${ty}) at index ${idx} in table for 'concat'`,
sf,
);
};
if (Array.isArray(tbl)) {
const out: string[] = [];
for (let k = i; k <= j; k++) {
const v = tbl[k - 1];
out.push(luaConcatElemToString(v, k));
}
return out.join(sep);
}
const out: string[] = [];
for (let k = i; k <= j; k++) {
const v = await luaGet(tbl, k, sf.astCtx ?? null, sf);
out.push(luaConcatElemToString(v, k));
}
return out.join(sep);
},
),
/**
* Inserts an element into a table at a specified position.
* @param tbl - The table to insert the element into.
* @param posOrValue - The position or value to insert.
* @param value - The value to insert.
*/
insert: new LuaBuiltinFunction(
async (
sf,
tbl: LuaTable | any[],
posOrValue: number | any,
value?: any,
) => {
if (Array.isArray(tbl)) {
// Since we're inserting/appending to a native JS array, we'll also convert the value to a JS value on the fly
// this seems like a reasonable heuristic
if (value === undefined) {
tbl.push(luaValueToJS(posOrValue, sf));
} else {
tbl.splice(posOrValue - 1, 0, luaValueToJS(value, sf));
}
return;
}
if (!(tbl instanceof LuaTable)) {
return;
}
let pos: number;
let v: any;
if (value === undefined) {
v = posOrValue;
pos = (await luaLenForTableLibAsync(sf, tbl)) + 1;
} else {
pos = posOrValue;
v = value;
}
const n = await luaLenForTableLibAsync(sf, tbl);
// Shift up: for k = n, pos, -1 do t[k+1] = t[k] end
for (let k = n; k >= pos; k--) {
const cur = await luaGet(tbl, k, sf.astCtx ?? null, sf);
await luaSet(tbl, k + 1, cur, sf);
}
await luaSet(tbl, pos, v, sf);
},
),
/**
* Removes an element from a table at a specified position.
* @param tbl - The table to remove the element from.
* @param pos - The position of the element to remove.
*/
remove: new LuaBuiltinFunction(
async (sf, tbl: LuaTable | any[], pos?: number) => {
if (Array.isArray(tbl)) {
const n = tbl.length;
const p = pos ?? n;
if (p < 1 || p > n) {
throw new LuaRuntimeError("position out of bounds", sf);
}
const idx = p - 1;
const v = tbl[idx];
tbl.splice(idx, 1);
return v;
}
if (!(tbl instanceof LuaTable)) {
return null;
}
const n = await luaLenForTableLibAsync(sf, tbl);
const p = pos ?? n;
if (p < 1 || p > n) {
throw new LuaRuntimeError("position out of bounds", sf);
}
const v = await luaGet(tbl, p, sf.astCtx ?? null, sf);
// Shift down: for k = p, n-1 do t[k] = t[k+1] end; t[n] = nil
for (let k = p; k < n; k++) {
const next = await luaGet(tbl, k + 1, sf.astCtx ?? null, sf);
await luaSet(tbl, k, next, sf);
}
await luaSet(tbl, n, null, sf);
return v;
},
),
/**
* Sorts a table.
* @param tbl - The table to sort.
* @param comp - The comparison function.
* @returns The sorted table.
*/
sort: new LuaBuiltinFunction(
async (sf, tbl: LuaTable | any[], comp?: ILuaFunction) => {
if (Array.isArray(tbl)) {
return await asyncQuickSort(tbl, async (a, b) => {
if (comp) {
return (await comp.call(sf, a, b)) ? -1 : 1;
}
return (a as any) < (b as any) ? -1 : 1;
});
}
if (!(tbl instanceof LuaTable)) {
return tbl;
}
const n = await luaLenForTableLibAsync(sf, tbl);
const values: any[] = [];
for (let i = 1; i <= n; i++) {
values.push(await luaGet(tbl, i, sf.astCtx ?? null, sf));
}
const cmp = async (a: any, b: any): Promise<number> => {
if (comp) {
const r = await luaCall(comp, [a, b], sf.astCtx ?? {}, sf);
return r ? -1 : 1;
}
const av = isTaggedFloat(a) ? a.value : a;
const bv = isTaggedFloat(b) ? b.value : b;
if (typeof av === "number" && typeof bv === "number") {
return av < bv ? -1 : 1;
}
if (typeof av === "string" && typeof bv === "string") {
return av < bv ? -1 : 1;
}
const ta = typeof av;
const tb = typeof bv;
throw new LuaRuntimeError(
`attempt to compare ${ta} with ${tb}`,
sf,
);
};
const sorted = await asyncQuickSort(values, cmp);
for (let i = 1; i <= n; i++) {
await luaSet(tbl, i, sorted[i - 1], sf);
}
return tbl;
},
),
/**
* Returns the keys of a table.
* Note: Space Lua specific
* @param tbl - The table to get the keys from.
* @returns The keys of the table.
*/
keys: new LuaBuiltinFunction((_sf, tbl: LuaTable | LuaEnv | any) => {
if (tbl.keys) {
return tbl.keys();
}
return Object.keys(tbl);
}),
/**
* Checks if a table (used as an array) contains a value.
* Note: Space Lua specific
* @param tbl - The table to check.
* @param value - The value to check for.
* @returns True if the value is in the table, false otherwise.
*/
includes: new LuaBuiltinFunction(
(sf, tbl: LuaTable | any[], value: LuaValue) => {
if (!tbl) {
return false;
}
if (tbl instanceof LuaTable) {
// Iterate over the table
for (const key of tbl.keys()) {
if (luaEquals(tbl.get(key), value)) {
return true;
}
}
return false;
}
if (Array.isArray(tbl)) {
return !!tbl.find((item) => luaEquals(item, value));
}
throw new LuaRuntimeError(
`Cannot use includes on a non-table or non-array value`,
sf,
);
},
),
/**
* Returns a new table from an old one, only with selected keys
* @param tbl a Lua table or JS object
* @param keys a list of keys to select from the table, if keys[0] is a table or array, assumed to contain the keys to select
* @returns a new table with only the selected keys
*/
select: new LuaBuiltinFunction(
(sf, tbl: LuaTable | Record<string, any>, ...keys: LuaValue[]) => {
// Normalize arguments
if (Array.isArray(keys[0])) {
// First argument is key array, let's unpack
keys = keys[0];
} else if (keys[0] instanceof LuaTable) {
keys = keys[0].toJSArray();
}
const resultTable = new LuaTable();
const setPromises: (void | Promise<void>)[] = [];
for (const key of keys) {
setPromises.push(resultTable.set(key, luaGet(tbl, key, null, sf)));
}
const promised = evalPromiseValues(setPromises);
if (promised instanceof Promise) {
return promised.then(() => resultTable);
}
return resultTable;
},
),
pack: new LuaBuiltinFunction((_sf, ...args: any[]) => {
const tbl = new LuaTable();
for (let i = 0; i < args.length; i++) {
tbl.set(i + 1, args[i]);
}
tbl.set("n", args.length);
return tbl;
}),
unpack: new LuaBuiltinFunction(
async (sf, tbl: LuaTable | any[], i?: number, j?: number) => {
i = i ?? 1;
if (j === undefined || j === null) {
j = Array.isArray(tbl)
? tbl.length
: await luaLenForTableLibAsync(sf, tbl);
}
const result: LuaValue[] = [];
for (let k = i; k <= j; k++) {
const v = Array.isArray(tbl)
? tbl[k - 1]
: await luaGet(tbl, k, sf.astCtx ?? null, sf);
result.push(v);
}
return new LuaMultiRes(result);
},
),
// Non-standard Lua functions
/**
* Finds an element in a table that matches a criteria function. Returns the first matching element.
* @param tbl - The table to search.
* @param criteriaFn - The criteria function.
* @param fromIndex - The index to start searching from.
* @returns Lua multi value of index, value, or nil if no element is found.
*/
find: new LuaBuiltinFunction(
async (
sf,
tbl: LuaTable | any[],
criteriaFn: ILuaFunction,
fromIndex = 1,
) => {
if (!tbl) {
return null;
}
const startIndex = fromIndex < 1 ? 1 : fromIndex;
const n = Array.isArray(tbl)
? tbl.length
: await luaLenForTableLibAsync(sf, tbl);
for (let i = startIndex; i <= n; i++) {
const val = await luaGet(tbl, i, sf.astCtx ?? null, sf);
if (await luaCall(criteriaFn, [val], sf.astCtx!, sf)) {
return new LuaMultiRes([i, val]);
}
}
return null;
},
),
});