Commit Graph
92 Commits
Author SHA1 Message Date
75cc79800f Space Lua: Align numeric/table semantics with Lua, align number formating, optimize loops allocations (#1823)
* Don't know what's going on with the Deno docker image, disabling the deno.lock file for now

* Another try

* Space Lua: Align numeric and table semantics with Lua

This change improves Space Lua compatibility with standard Lua 5.4,
focusing on numeric subtypes and table behavior. The test suite is
extended to lock in the expected semantics and should pass under both
Space Lua and a Lua interpreter.

SUMMARY OF CHANGES
------------------

Tighten Lua compatibility across evaluator and runtime:

- correct metamethod dispatch (`__index`, `__newindex`, `__call`,
  comparison metamethods),

- loop limits,

- raw metamethod lookups.

Rework numeric semantics to preserve Lua **integer** vs **float**
behavior:

- $0$ vs $0.0$ and $-0.0$,

- explicit zero kind representation, and

- updates arithmetic/bitwise coercions accordingly.

Improve parser correctness by rejecting **unary plus** with aligned Lua
errors and better parsing errors reporting.

Fix `stdlib` behavior to match that of Lua:

- `table` function `concat`, `insert`, `remove`, `sort` and `unpack`
  gain metamethod awareness and enforce Lua errors,

- `ipairs` iteration updated to stop on first nil and honor `__index`,

- `tonumber` updated to Luae conversion using `luaToNumberDetailed`,

- `math.modf` return corrected,

- `math.type` accuracy improvements for float/integer and $-0.0$,

- `math.pi` added.

Fix numeric subtypes for `/` and `^` operators so `math.type` matches
Lua results using tagging as well as unary `-`.

Expand test coverage:

- new `metamethods_test.lua` for Lua metamethod/operator semantics,

- extend arithmetic and length tests for zero-kind propagation and
  `rawlen` vs `__len` metamethod,

- Extend `math` test suite to test proper Lua alignment (`math.type` and
  more), and

- update context error expectations for Lua error messages.

RATIONALE
---------

Lua differs from JavaScript by having two numeric subtypes: **integer**
and **float**. Operators depend on the subtype: `+`, `-`, `*`, `//` and
`%` use integer mode when both operands are integers and float mode
otherwise. Bitwise operators require integers and `math.type(x)` reports
"integer" or "float". JavaScript has one numeric primitive type
(`number`) so a plain number value cannot record whether Lua considers
a value to be a float when the value has no fractional part (for example
$2.0$).

Lua also differs from "everything is IEEE 754 double" because the rules
are defined in terms of integer and float subtypes. Float operations
preserve IEEE 754 behavior including `NaN`, infinities and signed zero
($-0.0$) which affects results like $1/0.0$ versus $1/-0.0$. Integer
arithmetic does not preserve $-0$ and collapses it to $0$. Lua integer
arithmetic is exact within its integer range while JavaScript `number`
cannot exactly represent all integers in that range.

Lua numbers are integers or floats. Numeric strings coerce to integer or
float based on _lexical_ form. Each arithmetic operator selects the
result subtype from the operator rules and operand subtypes.  Integer
only operators (bitwise and `//` as integer division) require integer
representability. Mixed arithmetic promotes to float as needed.  Two
operators are **always float** typed: division (`/`) and exponentiation
(`^`) produce floats even if both operands are integers and even if the
numeric value has no fractional part. `math.type` reports that internal
subtype.

Tables are associative arrays and assigning `nil` removes a key. The
length operator `#` uses `__len` metamethod if present otherwise it uses
the raw length rule. `rawlen(table)` ignores `__len`. Without `__len`
Lua defines `#` as some boundary `N` such that `table[N]` is not `nil`
and `table[N+1]` is `nil`. If the table has holes (missing or `nil`
entries in the positive integer key sequence) the boundary may be non
unique so `#` is stable only for proper sequences without holes.

PERFORMANCE NOTES
-----------------

Numeric changes add small checks to preserve Lua integer and float
subtype semantics and avoid allocations except when the subtype would
otherwise be lost.

Some table operations may be slower due to stricter Lua 5.4 behavior
especially around length and sequence boundary handling which currently
requires extra metadata tracking and scans and cannot be avoided without
a completely different internal table representation.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Another try

* Replace `LuaFloatTag` plain objects with boxed Number for float tagging

* Restore pre-merge eval/numeric architecture and fix regressions

This commit restores the original branch architecture.

On top of the restored foundation, float-typed integer results (e.g.
`1.0 + 1.0` = `2.0`, `0.0 // 1.0` = `0.0`) are now correctly tagged via
`makeLuaFloat` so that `tostring` and `math.type` report them as floats.
The *unary minus* fast path for float literals and the `tonumber`
function also preserve float tagging.

Performance regressions from the merge are addressed by avoiding
`Number` boxing for non-integer floats (`3.14` needs no tag — it is
unambiguously float), adding *string key* fast paths in `LuaTable`
`has`/`rawGet`/`rawSet` to skip numeric normalization for the dominant
case, and inlining a `typeof` check in math standard library functions
to avoid function call overhead on plain numbers.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Replace boxed `Number` float tagging with plain tagged objects

* Replace `new Number()` boxing with plain tagged float objects for Lua
  float type tracking.

* Integer-valued floats that need type disambiguation are now
  represented as `{ value: number, isFloat: true }` instead of boxed
  `Number` objects with a symbol property.

* Pre-allocated singletons are used for positive and negative float
  zeros to avoid allocation entirely in common cases.

* Updated all detection, unwrapping, and coercion paths across
  `numeric.ts`, `runtime.ts`, `eval.ts`, `stdlib.ts`, and `stdlib/`
  modules to use the new `isTaggedFloat` type guard.

* Removed all `instanceof Number` checks.

* Deleted the `FloatKind` symbol and eliminated redundant helpers
  `isLuaFloat`, `isFloatTag`, `getZeroBoxKind` and `toPlainNumber` that
  became dead code.

* Simplified `math.type`, `luaToString`, `luaEquals`, `luaTypeName` and
  various other key normalization paths.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Add fast paths in `coerceNumericPair` for tagged float operands

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Fix copy/paste typo

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Avoid extra `LuaEnv` allocations in "For" and "ForIn" loops

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* [Space Lua] Reuse loop variable environment in `for` and `for-in` loops

Numeric `for` and generic `for-in` loops allocated a fresh `LuaEnv` on
every iteration to hold loop variables. But this is only necessary when
a closure inside the loop body captures the loop variable.

The optimization uses a two-level check computed at parse time. If no
function definition exists in the loop's subtree, environment reuse is
safe. When a function definition is present a deeper analysis walks the
block to determine whether any function body references the loop
variable names without them being shadowed by its own parameters. When
a closure captures a loop variable the loop fall back to per-iteration
allocation.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* [Space Lua] Format numbers using standard Lua rules

Standard Lua formats floats via C `sprintf("%.14g")` (14 significant
digits, scientific notation when shorter, exponent padded to 2 digits,
and a guaranteed `.0` suffix for integer-valued floats).

* Replace the old `luaFormatNumber` with JS `toPrecision(14)`-based
  implementation that reproduces this behavior.

* Integrates it so that `${}` expressions in the UI also display
  numbers correctly.

* Fixes tagged floats (`{ value, isFloat }`) were being stripped by
  `luaValueToJS` or matched as plain objects before reaching the number
  formatter. This caused `${}` expressions to render raw JS numbers.

Examples:

```
- ${tostring(2^63)}
- ${2^63}
- ${(2^63)}
```

All of the the above examples show correct `9.2233720368548e+18` now.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* [Space Lua] Fix `string.format` for floats and tagged numbers

Unwrap tagged floats before `printf`, handle `inf`/`-inf`/`-nan` in
`formatDouble`, and fix `%g` producing `0e+00` for zero. Hopefuly it's
enough to gain Lua formatting.

Tests:

```
- ${string.format("%.14g", 0.0)} - `0`
- ${string.format("%.14g", 1.0)} - `1`
- ${string.format("%.14g", 1/3)} - `0.33333333333333`
- ${string.format("%.14g", math.pi)} - `3.1415926535898`
- ${string.format("%.14g", 1e-10)} - `1e-10`
- ${string.format("%.14g", 1e18)} - `1e+18`
- ${string.format("%.14g", 2^63)} - `9.2233720368548e+18`
- ${string.format("%.14g", 2^53)} - `9.007199254741e+15`
- ${string.format("%.14g", 1.7976931348623e+308)} - `1.7976931348623e+308`
- ${string.format("%.14g", 5e-324)} - `4.9406564584125e-324`
- ${string.format("%.14g", 0/0)} - `-nan`
- ${string.format("%.14g", 1/0)} - `inf`
- ${string.format("%.14g", -1/0)} - `-inf`
```

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* [Space Lua] Add and integrate new `luaFormat` utility and test suite

* Add `luaFormat` string formatting function compatible with Lua, and
  integrate it across the codebase as a replacement for prior formatting
  approaches.

* Add extensive test suite.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Fix check

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* [Space Lua] Add `%a`/`%A` and `%q` format specifiers to `string.format`

Implement hexadecimal floating-point (`%a`/`%A`) and quoted literal
(`%q`) specifiers.

* Add `%a`/`%A` as IEEE 754 double decomposition with full flag, width
  and precision support.

* Add `%q` as producind valid Lua literals for strings, numbers,
  booleans and nil.

* Use `Math.PI` for `math.pi` to preserve full double precision.

* Remove Deno based test suite and replace it with native Lua test
  suite.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* [Space Lua] Add `%p` format specifier to `string.format`

In standard Lua, `%p` formats the internal C heap address of a value,
producing output like `0x55a3bc4e2f10`. It works on tables, functions,
threads, strings, and userdata (GC-ed objects). For `nil`, booleans, and
numbers it returns `(null)`.

In Space Lua, there are no *raw memory addresses* since the runtime is
JavaScript. Instead, `%p` assigns a *stable sequential integer* to each
object via a `WeakMap`, formatted as a 14-digit zero-padded hex value.
The key difference is that identifiers are deterministic and sequential
rather than random-looking heap addresses:

```lua
local t = {}

print(string.format("identifier: %p", t)) -- 0x00000000000001
print(string.format("the same:   %p", t)) -- 0x00000000000001
print(string.format("another:    %p", {}) -- 0x00000000000002
```

For strings, a regular `Map` is used so identical string content always
produces the same identifier.

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Revert merge changes to the deno.json

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Replace `interface` with `type`

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Remove `has_math()` relict function test

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Refactor loop to map

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Remove `Deno.remove("deno.lock")` weirdness

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

* Refactor: Early return undefined in `astNumberKind` instead of assigning

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>

---------

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
Co-authored-by: Zef Hemel <zef@zef.me>
2026-02-12 15:22:12 +01:00
Zef Hemel 2cea80f1de Swap out @luca/esbuild-deno-loader with "official" deno ES build loader (fixes building with Deno 2.6.5+ versions)
- Replaced `@luca/esbuild-deno-loader` with a new `build_deps.ts` file to manage build dependencies.
- Breaking change to plug-compile: --importmap no longer supported, use -c instead to pass in a deno.json (which can include a "import" map).
2026-02-06 08:12:11 +01:00
Zef Hemel 55bb0f0a0a Dependency bumps 2026-01-20 10:10:02 +01:00
Zef Hemel 3e1f03b2bb 2.4.1 2026-01-16 13:07:53 +01:00
Zef Hemel e4f869bb71 2.4.0 2026-01-16 13:04:17 +01:00
Zef Hemel a2c0c01ac7 preact upgrade 2026-01-15 09:27:06 +01:00
Zef Hemel 840e0c5fea Tuned the filterbox and fuse-based filtering
* Upgraded fuse library
* Removed the category support in the filterbox, it was buggy and hides
  command names
* Tweaked fuse parameters for (hopefully) better matches in the page
  picker (and everywhere where the filter box is used)
2026-01-15 09:17:34 +01:00
Zef Hemel fb3dd51bdb Inline content refactor 2026-01-12 17:19:23 +01:00
Zef Hemel 2470bea0b5 Start of rudementary refactoring of indexing Making index logic actually
testable. Three indexers migrated:
* page
* data
* item
2026-01-06 12:21:02 +01:00
Zef Hemel 0f57a03542 Remove full-text search from SB core
Now available as separate library:
https://github.com/silverbulletmd/basic-search/blob/main/PLUG.md
2025-11-24 16:33:17 +01:00
Zef Hemel 21ddcb462e 2.3.0 2025-11-24 10:48:00 +01:00
Matouš Jan FialkaandGitHub 3422a9fa20 Add Space Lua benchmarks based on Deno.bench aligned to Deno.test (#1649)
Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
2025-11-05 15:06:33 +01:00
Zef Hemel f44f00f4c2 2.2.1 2025-11-03 07:56:44 +01:00
Zef Hemel 93b3994e82 2.2.0 2025-11-03 07:54:18 +01:00
Zef HemelandGitHub 272f293ddb Optional client encryption (#1640)
Implements #1268
2025-10-22 15:35:37 +02:00
Zef Hemel aaaee4299a 2.1.9 2025-10-09 15:56:41 +02:00
Zef Hemel 17e838883b 2.1.8 2025-10-09 09:59:43 +02:00
Zef Hemel 5be6297271 2.1.7 2025-10-04 15:41:12 +02:00
Matouš Jan FialkaandGitHub 9967cf1fd1 Space Lua: Remove unary plus (#1585)
* Improve documentation on string to number conversion (plus minor fixes of typos)
* Update Lua grammar to disallow unary plus

Signed-off-by: Matouš Jan Fialka <mjf@mjf.cz>
2025-10-03 08:28:36 +02:00
Zef Hemel 9521c7605a 2.1.6 2025-09-30 11:08:37 +02:00
Zef Hemel 2b32ab65ea 2.1.5 2025-09-27 06:43:12 +02:00
Zef Hemel afb97a1aac 2.1.4 2025-09-26 12:19:32 +02:00
Zef Hemel a072b68c53 Fix lint 2025-09-25 15:53:10 +02:00
Zef Hemel 7a571836a6 2.1.3 2025-09-25 15:30:58 +02:00
Zef Hemel 60dbb13d4a 2.1.2 2025-09-25 13:19:49 +02:00
Zef Hemel 9f3087a665 2.1.1 2025-09-25 10:41:46 +02:00
Zef Hemel f545497d0d 2.1.0 2025-09-25 10:16:50 +02:00
Zef Hemel dd9e4582dd Move built client files to client_bundle folder to make embedding them into the server binary cleaner. 2025-09-25 09:51:05 +02:00
Zef Hemel f4336bf09f Code restructuring: eliminated the top-level lib/ directory 2025-09-23 17:00:34 +02:00
Zef HemelandGitHub a524c64884 Go backend (#1555)
* Replaces old Deno server with a Go implementation
* Restructured the code base accordingly
2025-09-19 13:23:20 +02:00
Zef HemelandGitHub dc2987a3f9 Sync engine rearchitecture (#1516)
* Don't require auth for .ping
* Finally introducing loggers
* Remove file list caching from disk_space_primitives
I don't think this is required anymore with the sync model
* Simplification of space primitives and more tests
* More reliable online status
* Add .zed settings
* Remove selfUpdate argument from `writeFile` no longer used
- Add `sync` config schema and document sync options
- Move sync config handling to client boot logic
- Move BootConfig type to ui_types.ts
- Remove sync env vars from server
- Update documentation for sync and architecture
* Improve sync logic and proxy routing for full sync state
* Add BoxProxy for late-bound client and improve Lua boot
- Add isMobile syscall for editor
* Add concurrency to sync process and log indexing duration
* Better service worker lifecycle management
2025-09-12 14:43:57 +02:00
Zef Hemel 644e027abe Fixes #1515
Replaces hacky way of building two binaries for ARM and Intel and putting the right one into the image by using a multi-stage build. You can now build SB without even installing Deno.

Also adds a HEALTCHECK
2025-09-01 09:09:16 +02:00
Zef Hemel 7571ad1972 2.0.0 2025-08-29 15:38:06 +02:00
Zef Hemel f5cd8ceb8c 2.0.0-rc3 2025-08-29 15:32:39 +02:00
Zef Hemel fdf5a3acf6 2.0.0-rc2 2025-08-29 15:31:18 +02:00
Zef Hemel 2e16bc0663 2.0.0-rc1 2025-08-29 15:23:09 +02:00
Zef HemelandGitHub fe5dc80a14 Expose file system API on /.fs instead of root (#1507)
Fixes #1506
2025-08-27 09:58:11 +02:00
aa0b95b31d The one about refs (#1496)
* [Major] Introduce a new `Ref` and `Path` type

This introduces a new `Ref` and `Path` type, which try to be as lightweight and rigorous as possible. More description in the PR

* [Minor] Updated Syntax

* [Minor] Updated Type

* [Minor] Renamed file

* [Minor] Updated type

* [Minor] Updated Syntax

* [Major] Rename css classes for wiki links

Previously all the parts of a wiki link were tagged with `sb-wiki-link-page`. This didn't make a whole lot of sense. Also the tag `sb-wiki-link` was could never render, because of the way `HighlightStyle` works. Now every part of a wiki link will be tagged with `sb-wiki-link` (i.e. the link, the alias, the dimensions). Invalid and missing links will be styled separately in a later commit.

* [Minor] Adjust to new API

* [Minor] Small changes

* [Minor] Function and file renames

* [Major] Syscall adjusments

Change behavior of `getCurrentPath` to actually return what's now considered a `Path`. `getCurrentPage(Meta)` now also correctly returns for documents. We could consider a rename to get rid of the `page` association

* [Major] Rework `Client`

Mostly minor adjustments for the new API. Some major changes in `navigateWithinPage` and some unecessary checks removed now

* [Major] Errors on navigate

This now throws an error when the user tries to navigate to a link that is invalid

* [Major] Minor adjustments and remove some old bulk

This does the typicall adjustments to the new API. A thing to consider is that invalid links aren't indexed. Also removed old code about template and query stuff. And reused a regex instead of copy pasting it.

* [Minor] Cleanup editor ui

* [Minor] Adjust to new API

* [Minor] Adjust to new API and reduce ugly regex

* [Minor] New API

* [Minor] Rework link cm plugin

* [Major] Rework wiki links

A lot of code cleanup, reworked for new APIs and also introduces "invalid links"

* [Minor] Part of the editor ui rework

* [Major] Adjust to new API

This also removes the `looksLikePathWithExtension` check. I don't think it was necessary.

* [Minor] PArt of editor ui rewrite

* [Minor] Adjust to new API

* [Major] Adjust renderer to new API

Invalid links now link to `#`

* [Major] Rewrite navigator

This still needs major rework, currently in a "got it working" state

* [Minor] Adjust file name

* [Minor] Mistake during rebase

* [Minor] Navigate to end of header

* [Minor] Remove `tweakEditorDOM` and use proper CM extensions

* [Major] Heavily clean up the navigation code

* [Minor] Make `PageNavigator` closable again

* [Minor] Small fixes to reduce errors bubbling up

* [Minor] Show error message if navigate fails

* [Major] Remove legacy `navigate({ kind = ` syntax

This removes the old navigation syntax. It's still supported but the user will be warned then using it. I also added some validation to editor.navigate for user provided refs.

* [Minor] Make notifications errors

* [Minor] Don't fail if header isn't found

* [Minor] Typo

* [Major] Remove cleanPageRef

* [Minor] Small bug

* [Minor] Align behavior between render and editor

* [Major] Remove `PageRef`. Pretty sure it's obsolete

* [Major] Remove pos from header as it's an invalid ref

* [Minor] Align with original behavior

* [Minor] Get tests working again

* [Minor] check that indexPage is valid

* [Major] Rewrite `resolve.ts` file

* [Minor] More tests, more fixes

* [Minor] Don't just append .md this is weird

* [Minor] More edge cases

* [Minor] Version bumping hono somehow fixes the `/` case

* [Minor] Smaller TODOs

* [Major] We can be a littler looser on the restrictions

* [Major] Change header indexing, to index with pos

* [Minor] More fixes for headers

* [Minor] Rename for clarity

* [Minor] Change more regexes to named groups

* [Major] Inline content rewrite

* [Minor] Fix TODO

* [Minor] Bump index version

* [Major] New Space Primitive to check paths

* [Minor] Fix #1214

* [Minor] Fix #1169

* [Minor] Remove TODO

This check is probably not needed anymore. Unsure. May look into it later

* [Minor] small bug

* [Minor] Not a good rebase without me needing to fix it afterwards

* [Minor] Improve fallback for file upload

* [Minor] Some fixes on the for resolving

* [Minor] Removed doubled test case

* [Minor] Resolve #860

* [Minor] Turn flash notification into an error

* [Minor] Change handling of `.` paths

* [Minor] Show empty URL for index page

* [Minor] Push correct ref to history

* [Major] Capture mini editor events in capture phase

This fixes the issue were when pressing arrowup or arrowdown changed the cursor position inside the editor

* [Major] Add prefix to `FilterList`

This now fixes issues were if you had a prefixed page you  con't properly search for it, because the search considered the prefix as part of the name

* [Minor] A little rigor for empty headers

* [Major] Fix #1404

* [Minor] `parseInt` returns `NaN` handle that

* [Minor] Use `fileName`

* [Minor] Fix comment

* [Minor] Fixes

* [Major] disallow `foo.png.md` and `/../`, etc. cases

* [Major] Change name of conflicted pages

* [Minor] Remove unecessary check

* [Minor] Remove some split code

* [Minor] Allow deletion of invalid files

* [Minor] Bug in the regex

* [Minor] Give some possibility of fixing incorrect names

* [Minor] Tiny bug with header completion

* [Major] Docs

* [Minor] Ignore errors for navigation inside of a page

This could be a problem when SB e.g. cached a scrollTop for a page, which, because e.g. the file is now deleted, doesn't exist anymore and thus CM throws an error, when trying to navigate

* Update conflicted copy regex in Maintenance.md

---------

Co-authored-by: Zef Hemel <zef@zef.me>
2025-08-24 16:16:19 +02:00
Zef Hemel efdb54d282 Fix deno task install
Not sure why this worked before...
2025-06-18 17:43:16 +02:00
Zef Hemel cc3aece55c Don't include sourcemap in production build 2025-06-18 15:45:25 +02:00
Zef Hemel 40c2bd82d7 Dependency bumps 2025-06-18 11:42:01 +02:00
Zef Hemel a2917a9d10 Dependency bumps 2025-06-18 11:01:41 +02:00
Zef Hemel 6d8deb969e Migrate from esm.sh to npm: packages 2025-06-18 10:49:19 +02:00
Zef Hemel d92eeb85c2 2.0.0-pre4 2025-06-18 10:11:42 +02:00
Zef Hemel 42bd980419 2.0.0-pre3 2025-06-18 10:01:24 +02:00
Zef Hemel ac85013908 2.0.0-pre2 2025-06-18 09:47:20 +02:00
Zef Hemel 23dcb73f44 2.0.0-pre1 2025-06-18 09:39:38 +02:00
Zef Hemel 450ef50df7 Cleanup: more cleanup and moving stuff around 2025-06-17 16:12:18 +02:00
Zef Hemel c3672056fb Eliminate $lib 2025-06-16 18:53:21 +02:00
Zef Hemel ea41d195b2 Minor code tweaks 2025-06-12 21:12:50 +02:00