Items from `index.tag` are stored as plain JS objects when no metatable
is defined. The `group by` clause wraps rows in `LuaTable` (`key`,
`group`) but leaves the individual `group` items as plain JS objects.
This causes `t.keys is not a function` errors when passing `row.group`
to APIs like `template.each` that expect `LuaTable` instances.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
Evaluate all `order by` expressions once per item in a decorate pass,
then compare cached keys during sort. Reduces Lua eval calls from
*O(kn log n)* to *O(kn)* where *k* is the number of sort keys.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
- Add iterative async stable merge sort (*O(n log n)*).
- Replace Quicksort with the new merge sort (kept Quicksort for tables).
- Update documentation (reorganize for logical flow, add SWO section
with invalid comparator example that really throws).
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* support for custom comparators in LIQ `order by` clauses via the
`using` keyword.
* `using` is mutually exclusive with `asc`/`desc` and accepts either
a *named function* or an *inline anonymous function*. The comparator
receives two values and returns true if the first should come before the
second.
* `nulls first`/ `nulls last` works independently of `using`.
* `using` is now a reserved keyword (`kw`; this cannot be avoided but
the chance to collide with user scripts is negligable),
* documentation updated + polished a little
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
Implements `filter(where <cond>)` for aggregate functions in LIQ
queries which allows per-row conditional filtering within grouped
aggregates. Also extends test suite and documentation.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
Extend the `order by` clause with optional `nulls first` and `nulls
last` modifiers following standard SQL semantics. Each sort key can
independently specify where `nil` values should appear in the result. By
default, ascending sorts place nulls last and descending sorts place
nulls first. An explicit nulls first or nulls last overrides this
default. The `asc` keyword is also now accepted as an explicit
alternative to the default ascending direction. Documentation and tests
were updated accordingly.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
Check for already-queryable objects (e.g. `DataStoreQueryCollection`)
before `LuaTable` extraction or `luaValueToJS` conversion, preventing
them from being destroyed by the collection normalization pipeline.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
- Fix `group by` key building: populate array indices from
`keyParts` instead of only from `keyRecord`, which was empty for
some expressions (e.g. `tags[1]`).
- Fix collection handling: treat record-like `LuaTables` (no array part)
as items instead of extracting zero items.
- Update test suite.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Fix `LUA_SQL_NULL` leaking into `order by` comparisons
Note: SQL `NULLS LAST` is now default behaviour!
* Fix `objectVariable` unset for empty groups
* Fix non-grouped `having` used wrong environment builder
Note: Queries similar to the following one now works correctly:
```sql
${query[[
from
tags.page
having
name == 'index'
]]}
```
* Fix `generateKey` did not handle `LUA_SQL_NULL`
* Refactor/deduplicate duplicated `order by` comparator
* Add `isSqlNull` helper
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
1. Fix `order by` inability to access `select` output aliases and
aggregate functions after `group by`.
2. Introduce `LUA_SQL_NULL` sentinel to preserve column ordering when
`select` expressions evaluate to `nil`.
Note: In Lua, setting a table key to `nil` deletes it so rows with
`nil` values lose those keys and produced inconsistent insertion
order across result tables.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* [LIQ] Extend with `group by` and `having` support
Also update documentation with extensive examples and add some parser
tests.
Note: `make generate` JS files included.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Merge `liq-add-support-for-aggregators` branch
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Fix link, remove `_` example
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Wrap example LIQ queries inside `query [[ .. ]]` and replace `sql` code block type to `lua`
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Example self-consistency fix
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Fixed some queries
* Add experimental tags to LIQ aggregation/grouping
---------
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
Co-authored-by: Zef Hemel <zef@zef.me>
The three binary-packing functions from Lua are now implemented in
a dedicated `string_pack.ts` module and wired into `stringApi`.
Format parsing follows the exact option set of Lua:
* signed and unsigned integers of configurable width
(`b`/`B`/`h`/`H`/`l`/`L`/`j`/`J`/`T`/`i`/`I`),
* IEEE 754 single and double precision floats (`f`/`d`/`n`),
* fixed-length strings (`c`),
* length-prefixed strings (`s`),
* zero-terminated strings (`z`),
* padding bytes (`x`),
* alignment padding (`X`), and
* endianness/alignment modifiers (`<`/`>`/`=`/`!`).
`string.packsize` rejects variable-length formats (`s` and `z`) as Lua
requires.
`string.unpack` returns all unpacked values followed by the
next position integer, matching the Lua contract exactly.
Test suite extended in separates `string_pack_test.lua` file. Passes
both Lua and Space Lua execution.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
`table.pack` now routes element writes through `luaSet` so `__newindex`
metamethods are honoured, matching the exact Lua 5.4 semantics. The `n`
field is still written with `rawSet`, also matching Lua.
`table.unpack` now returns an empty `LuaMultiRes` instead of `null` for
an empty range `(i > j)`. Returning `null` would inject a single `nil`
argument at the call site; an empty `LuaMultiRes` correctly contributes
zero values, matching the exact Lua semantics.
The global `unpack` shim (for Lua 5.1 compatibility) has been removed
from `luaBuildStandardEnv`. Users who need the short alias can set it up
in `space-script` with a single line stating `unpack = table.unpack`.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
Returning `1` when the comparator yields `false`/`nil` declares `a > b`
for equal elements. `asyncQuickSort` tolerates this today but any future
sort algorithm change could corrupt results. Changed to `0` so ties are
neutral.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
Implement `table.move(a1, f, e, t [,a2])`.
The implementation copies elements `a1[f..e]` to `a2[t..t+(e-f)]`
defaulting `a2` to `a1` and returning `a2` as follows:
* empty ranges (`e < f`) are "no-op",
* for in-table copies where `t > f` loop backwards to prevent unread
source slots from being clobbered before they are read (note: the only
case where overlap is destructive,
* cross-table copies always run forwards since there is no aliasing.
Add test suite that covers:
* non-overlapping copies,
* cross-table copies,
* both overlap directions,
* same position identity,
* empty ranges, and
* single element moves.
Each of the above is tested with both scalar and nested-table values.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
`tostring()` was delegating to `luaToString()` which calls `__tostring`
with a dead stack frame, breaking metamethods that call back into
globals. Now `tostringFunction` receives the live stack frame, looks up
`__tostring` via `rawGet`, calls it through `luaCall` with the live
frame, and validates the return is a string.
Also add tests covering the basic case, the nested-call case, and the
error path for a non-string return.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
Replace raw `String(v)` calls with `luaFormatNumber` in the concatenation
operator and `table.concat`, so that floats are formatted according to Lua
semantics rather than JavaScript's. This fixes cases like `10.8*22..""`
producing `237.60000000000002` instead of `237.6`, and ensures
integer-valued floats carry the `.0` suffix as Lua requires.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Add Lua *pattern matching engine* (`pattern.ts`) that directly ports
`lstrlib.c` semantics.
* Integrate the new engine into the `string` API for `find`, `match`,
`gmatch`, and `gsub` functions.
* Add a pattern test suite (`pattern_test.lua`) covering character
classes, repetition quantifiers, edge cases as well as error
conditions.
* Refactor `string_test.lua` to remove duplicated pattern tests that
are now covered by the pattern test suite.
* Fix `string.rep` separator support.
* Fix `string.sub` index handling.
* Remove the obsolete `string_test2.lua` file.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Aligned with Lua 5.4 `os.date` semantics: `!` prefix selects UTC,
absent format defaults to `%c`, `*t` returns a table with `isdst`
only for local time, invalid specifiers raise an error.
* All `strftime` specifiers documented in the Lua 5.4 reference manual
are supported, including ISO week-based year (`%G`/`%g`/`%V`) and
composite formats.
* Extensive test suite uses the Go reference timestamp
`2006-01-02 15:04:05 UTC` (epoch `1136214245`) where every field -
year, month, day, hour, minute, second — has a distinct value.
Tests cover UTC and local modes, AM/PM, Sunday edge cases for
`%u`/`%w`, ISO week-year boundaries, `%%` escaping, `*t` table fields,
and invalid specifier rejection.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Don't know what's going on with the Deno docker image, disabling the deno.lock file for now
* Another try
* Space Lua: Align numeric and table semantics with Lua
This change improves Space Lua compatibility with standard Lua 5.4,
focusing on numeric subtypes and table behavior. The test suite is
extended to lock in the expected semantics and should pass under both
Space Lua and a Lua interpreter.
SUMMARY OF CHANGES
------------------
Tighten Lua compatibility across evaluator and runtime:
- correct metamethod dispatch (`__index`, `__newindex`, `__call`,
comparison metamethods),
- loop limits,
- raw metamethod lookups.
Rework numeric semantics to preserve Lua **integer** vs **float**
behavior:
- $0$ vs $0.0$ and $-0.0$,
- explicit zero kind representation, and
- updates arithmetic/bitwise coercions accordingly.
Improve parser correctness by rejecting **unary plus** with aligned Lua
errors and better parsing errors reporting.
Fix `stdlib` behavior to match that of Lua:
- `table` function `concat`, `insert`, `remove`, `sort` and `unpack`
gain metamethod awareness and enforce Lua errors,
- `ipairs` iteration updated to stop on first nil and honor `__index`,
- `tonumber` updated to Luae conversion using `luaToNumberDetailed`,
- `math.modf` return corrected,
- `math.type` accuracy improvements for float/integer and $-0.0$,
- `math.pi` added.
Fix numeric subtypes for `/` and `^` operators so `math.type` matches
Lua results using tagging as well as unary `-`.
Expand test coverage:
- new `metamethods_test.lua` for Lua metamethod/operator semantics,
- extend arithmetic and length tests for zero-kind propagation and
`rawlen` vs `__len` metamethod,
- Extend `math` test suite to test proper Lua alignment (`math.type` and
more), and
- update context error expectations for Lua error messages.
RATIONALE
---------
Lua differs from JavaScript by having two numeric subtypes: **integer**
and **float**. Operators depend on the subtype: `+`, `-`, `*`, `//` and
`%` use integer mode when both operands are integers and float mode
otherwise. Bitwise operators require integers and `math.type(x)` reports
"integer" or "float". JavaScript has one numeric primitive type
(`number`) so a plain number value cannot record whether Lua considers
a value to be a float when the value has no fractional part (for example
$2.0$).
Lua also differs from "everything is IEEE 754 double" because the rules
are defined in terms of integer and float subtypes. Float operations
preserve IEEE 754 behavior including `NaN`, infinities and signed zero
($-0.0$) which affects results like $1/0.0$ versus $1/-0.0$. Integer
arithmetic does not preserve $-0$ and collapses it to $0$. Lua integer
arithmetic is exact within its integer range while JavaScript `number`
cannot exactly represent all integers in that range.
Lua numbers are integers or floats. Numeric strings coerce to integer or
float based on _lexical_ form. Each arithmetic operator selects the
result subtype from the operator rules and operand subtypes. Integer
only operators (bitwise and `//` as integer division) require integer
representability. Mixed arithmetic promotes to float as needed. Two
operators are **always float** typed: division (`/`) and exponentiation
(`^`) produce floats even if both operands are integers and even if the
numeric value has no fractional part. `math.type` reports that internal
subtype.
Tables are associative arrays and assigning `nil` removes a key. The
length operator `#` uses `__len` metamethod if present otherwise it uses
the raw length rule. `rawlen(table)` ignores `__len`. Without `__len`
Lua defines `#` as some boundary `N` such that `table[N]` is not `nil`
and `table[N+1]` is `nil`. If the table has holes (missing or `nil`
entries in the positive integer key sequence) the boundary may be non
unique so `#` is stable only for proper sequences without holes.
PERFORMANCE NOTES
-----------------
Numeric changes add small checks to preserve Lua integer and float
subtype semantics and avoid allocations except when the subtype would
otherwise be lost.
Some table operations may be slower due to stricter Lua 5.4 behavior
especially around length and sequence boundary handling which currently
requires extra metadata tracking and scans and cannot be avoided without
a completely different internal table representation.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Another try
* Replace `LuaFloatTag` plain objects with boxed Number for float tagging
* Restore pre-merge eval/numeric architecture and fix regressions
This commit restores the original branch architecture.
On top of the restored foundation, float-typed integer results (e.g.
`1.0 + 1.0` = `2.0`, `0.0 // 1.0` = `0.0`) are now correctly tagged via
`makeLuaFloat` so that `tostring` and `math.type` report them as floats.
The *unary minus* fast path for float literals and the `tonumber`
function also preserve float tagging.
Performance regressions from the merge are addressed by avoiding
`Number` boxing for non-integer floats (`3.14` needs no tag — it is
unambiguously float), adding *string key* fast paths in `LuaTable`
`has`/`rawGet`/`rawSet` to skip numeric normalization for the dominant
case, and inlining a `typeof` check in math standard library functions
to avoid function call overhead on plain numbers.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Replace boxed `Number` float tagging with plain tagged objects
* Replace `new Number()` boxing with plain tagged float objects for Lua
float type tracking.
* Integer-valued floats that need type disambiguation are now
represented as `{ value: number, isFloat: true }` instead of boxed
`Number` objects with a symbol property.
* Pre-allocated singletons are used for positive and negative float
zeros to avoid allocation entirely in common cases.
* Updated all detection, unwrapping, and coercion paths across
`numeric.ts`, `runtime.ts`, `eval.ts`, `stdlib.ts`, and `stdlib/`
modules to use the new `isTaggedFloat` type guard.
* Removed all `instanceof Number` checks.
* Deleted the `FloatKind` symbol and eliminated redundant helpers
`isLuaFloat`, `isFloatTag`, `getZeroBoxKind` and `toPlainNumber` that
became dead code.
* Simplified `math.type`, `luaToString`, `luaEquals`, `luaTypeName` and
various other key normalization paths.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Add fast paths in `coerceNumericPair` for tagged float operands
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Fix copy/paste typo
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Avoid extra `LuaEnv` allocations in "For" and "ForIn" loops
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* [Space Lua] Reuse loop variable environment in `for` and `for-in` loops
Numeric `for` and generic `for-in` loops allocated a fresh `LuaEnv` on
every iteration to hold loop variables. But this is only necessary when
a closure inside the loop body captures the loop variable.
The optimization uses a two-level check computed at parse time. If no
function definition exists in the loop's subtree, environment reuse is
safe. When a function definition is present a deeper analysis walks the
block to determine whether any function body references the loop
variable names without them being shadowed by its own parameters. When
a closure captures a loop variable the loop fall back to per-iteration
allocation.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* [Space Lua] Format numbers using standard Lua rules
Standard Lua formats floats via C `sprintf("%.14g")` (14 significant
digits, scientific notation when shorter, exponent padded to 2 digits,
and a guaranteed `.0` suffix for integer-valued floats).
* Replace the old `luaFormatNumber` with JS `toPrecision(14)`-based
implementation that reproduces this behavior.
* Integrates it so that `${}` expressions in the UI also display
numbers correctly.
* Fixes tagged floats (`{ value, isFloat }`) were being stripped by
`luaValueToJS` or matched as plain objects before reaching the number
formatter. This caused `${}` expressions to render raw JS numbers.
Examples:
```
- ${tostring(2^63)}
- ${2^63}
- ${(2^63)}
```
All of the the above examples show correct `9.2233720368548e+18` now.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* [Space Lua] Fix `string.format` for floats and tagged numbers
Unwrap tagged floats before `printf`, handle `inf`/`-inf`/`-nan` in
`formatDouble`, and fix `%g` producing `0e+00` for zero. Hopefuly it's
enough to gain Lua formatting.
Tests:
```
- ${string.format("%.14g", 0.0)} - `0`
- ${string.format("%.14g", 1.0)} - `1`
- ${string.format("%.14g", 1/3)} - `0.33333333333333`
- ${string.format("%.14g", math.pi)} - `3.1415926535898`
- ${string.format("%.14g", 1e-10)} - `1e-10`
- ${string.format("%.14g", 1e18)} - `1e+18`
- ${string.format("%.14g", 2^63)} - `9.2233720368548e+18`
- ${string.format("%.14g", 2^53)} - `9.007199254741e+15`
- ${string.format("%.14g", 1.7976931348623e+308)} - `1.7976931348623e+308`
- ${string.format("%.14g", 5e-324)} - `4.9406564584125e-324`
- ${string.format("%.14g", 0/0)} - `-nan`
- ${string.format("%.14g", 1/0)} - `inf`
- ${string.format("%.14g", -1/0)} - `-inf`
```
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* [Space Lua] Add and integrate new `luaFormat` utility and test suite
* Add `luaFormat` string formatting function compatible with Lua, and
integrate it across the codebase as a replacement for prior formatting
approaches.
* Add extensive test suite.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Fix check
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* [Space Lua] Add `%a`/`%A` and `%q` format specifiers to `string.format`
Implement hexadecimal floating-point (`%a`/`%A`) and quoted literal
(`%q`) specifiers.
* Add `%a`/`%A` as IEEE 754 double decomposition with full flag, width
and precision support.
* Add `%q` as producind valid Lua literals for strings, numbers,
booleans and nil.
* Use `Math.PI` for `math.pi` to preserve full double precision.
* Remove Deno based test suite and replace it with native Lua test
suite.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* [Space Lua] Add `%p` format specifier to `string.format`
In standard Lua, `%p` formats the internal C heap address of a value,
producing output like `0x55a3bc4e2f10`. It works on tables, functions,
threads, strings, and userdata (GC-ed objects). For `nil`, booleans, and
numbers it returns `(null)`.
In Space Lua, there are no *raw memory addresses* since the runtime is
JavaScript. Instead, `%p` assigns a *stable sequential integer* to each
object via a `WeakMap`, formatted as a 14-digit zero-padded hex value.
The key difference is that identifiers are deterministic and sequential
rather than random-looking heap addresses:
```lua
local t = {}
print(string.format("identifier: %p", t)) -- 0x00000000000001
print(string.format("the same: %p", t)) -- 0x00000000000001
print(string.format("another: %p", {}) -- 0x00000000000002
```
For strings, a regular `Map` is used so identical string content always
produces the same identifier.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Revert merge changes to the deno.json
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Replace `interface` with `type`
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Remove `has_math()` relict function test
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Refactor loop to map
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Remove `Deno.remove("deno.lock")` weirdness
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Refactor: Early return undefined in `astNumberKind` instead of assigning
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
---------
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
Co-authored-by: Zef Hemel <zef@zef.me>
* Space Lua: Refactor control flow signals
Removes "return throws an exception" failure mode. @zef :)
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Remove unused function
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Remove another unused function
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Remove unused `LuaBreak`/`LuaReturn` exception control flow
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
---------
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Space Lua: `<close>` attribute, `__close` metamethod and extensive tests
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Space Lua: Align arithmetical error messages with Lua 5.4
**Fixes:**
- Tighten `tonumber` semantics to accept hex floats like `0x1.2` while
keeping plain hex as integers.
- Improve Lua compatibility by fixing relational `>`/`>=` operand
ordering to preserve correct error type direction.
**Changes:**
- Update concatenation errors to include the operand type.
- Add `LuaStringCoercionError` from numeric coercion and handle it
in `luaOp()` to emit correct messages (or fallback).
- Raise
> `bad argument #2 to 'tonumber' (base out of range)`
for invalid bases.
- Update test suite
**Notes:**
- The `arithmetic_test.lua` test suite now passes if run with Lua 5.4.
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Run `make fmt`
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Rename `assert_throws` to `assertThrows` in Lua test suite
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
* Replace `LuaStringCoercionError` class with sentinel constant
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
---------
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
Zef Hemel