mirror of
https://github.com/wu736139669/hapi.git
synced 2026-11-17 03:20:40 +00:00
feat(android): generalize generated media blocks to video/audio/file (#2162)
* feat(android): render chat markdown math with offline KaTeX Port the web math pipeline to Android chat markdown. A new MarkdownMath pre-pass in :core:protocol extracts math with web semantics (remark-math singleDollarTextMath:false + remark-latex- bracket-math): $$ display flows, $$ text math only when the closer run equals the opener and stays inside the paragraph, single $ never triggers math (currency stays prose), \(..\)/\[..\] bracket pairs, backslash-escaped delimiters stay literal, and code spans, fences, links and link reference definitions are fully protected. Matches are lifted behind private-use placeholder tokens and re-inserted as MathInlineNode/MathBlockNode custom nodes, hoisting display math out of paragraphs while blockquote/list containers survive the reparse. 37 JVM tests mirror the web suite's cases. Rendering is a shared offline WebView (ui/markdown/MathWebView.kt) fed a self-built HTML document (MathDocument.kt) against bundled assets/katex/ (KaTeX 0.16.45, same version as web). The view is network-blocked (blockNetworkLoads + shouldInterceptRequest refuses http/https/ws/wss) with file/content access disabled; asset URLs still resolve through the file:///android_asset/katex/ base URL. Theme color and font size are injected per render and the document reports its height through a JS bridge (ResizeObserver + onPageFinished re-measure). KaTeX failures follow the web's three-tier semantics, degrading to monospace raw TeX. Inline \(..\) and single-paragraph $$ math degrades to monospace TeX for v1: AnnotatedString cannot host a WebView and one WebView per fragment is too costly in chat lists. The parser still emits MathInlineNode so a future renderer can upgrade inline rendering without touching the extraction layer. Co-Authored-By: Claude Code <noreply@anthropic.com> via [HAPI](https://hapi.run) Co-Authored-By: HAPI <noreply@hapi.run> * fix(android): convert CSS pixels correctly at the math WebView boundary The JS bridge reports scrollHeight in CSS pixels, but KaTeXMathBlock treated the value as Android physical pixels and divided it by the screen density when sizing the modifier: on a density-3 device a reported 90 CSS px block collapsed to 30dp instead of ~90dp. The font conversion had the opposite mismatch — 16sp chat text went through TextUnit.toPx() (physical pixels) and was injected as `font-size: 48px` CSS, making formulas render ~3x too large on the same screen. Define the two boundary conversions explicitly as pure functions next to the document builder and unit-test them: - spFontToCssPx: CSS px and sp are both density-independent, so only the user's font-scale preference crosses the boundary; density never enters (16sp stays 16 CSS px at any density, 21 at fontScale 1.3). - cssPxToDp: reported CSS px map 1:1 to dp at the default WebView scale, multiplied by WebView.scale when the view is zoomed. The document now also pins the viewport with `width=device-width, initial-scale=1`, making the 1 CSS px = 1 dp default scale an explicit property of the document rather than a WebView default, with a regression test asserting the meta tag. Co-Authored-By: Claude Code <noreply@anthropic.com> * fix(android): refresh the math WebView height callback across spec changes KaTeXMathBlock keyed its height state with remember(spec) but assigned MathWebView.onHeightChanged inside AndroidView.factory, which runs only when the Android view is created. The callback therefore captured the initial spec's state object: when LaTeX, theme color or font size changed, remember(spec) minted a new state while AndroidView retained the existing view and its stale callback, so every measurement from the reloaded document updated the orphaned state and the current block stayed at its 22dp minimum. Keep one stable holder (MathViewHeightState) remembered unkeyed for the mounted view, reset it explicitly in onRender when the spec changes, and re-point the view's callback plus reset the height inside AndroidView.update before rendering — the creation-time callback then always feeds live state. JVM tests drive the holder through spec changes on one mounted view (the pre-fix wiring cannot compile against this shape: the factory-captured remember(spec) state is exactly the orphan the holder removes). Co-Authored-By: Claude Code <noreply@anthropic.com> * fix(protocol): bound dollar flow math closing to the enclosing container findFlowClose kept scanning after the opening line left its container and accepted an unprefixed closing run, so `> $$\n> x\n\nOrdinary prose\n\n$$\ny\n$$` paired the quoted opener with the later top-level `$$`: the prose was swallowed into the formula and the final `$$` lost its own opener. remark-math (verified against micromark-extension-math, whose math flow is `concrete` and dies with its container) renders the quote's math, the prose and a separate top-level equation instead. Bound matching to the opening container, distinguishing container shapes: a closer is only accepted from a line that still carries the container prefix; a blank line ends a blockquote (a following `> $$` opens a NEW quote, while a `>`-only line is an in-quote blank the container survives); a list item may resume after blank lines (loose lists), so blanks are skipped and the next content line decides; an unrecognized prefix is plain <= 3 space indentation, not a container, and keeps the unbounded scan (blank lines and fences stay math content). Regression tests: the quoted example (prose stays prose, the later equation keeps its own opener), a list-container variant, a blank-line/new-quote guard, plus two in-container close locks — the first three fail on the pre-fix scan. Co-Authored-By: Claude Code <noreply@anthropic.com> * feat(android): render mermaid fences as offline diagrams Port the web MermaidDiagram component to Android chat markdown. A fence whose language is exactly `mermaid` (case-sensitive, like the web's MARKDOWN_COMPONENTS_BY_LANGUAGE key) now dispatches to MermaidDiagramView instead of the plain code block; every other fence keeps the CodeBlock path. Rendering runs in a hardened offline WebView fed a self-built HTML document against bundled assets/mermaid/mermaid.min.js (Mermaid 11.14.0, curl-fetched, sha256-identical to the web app's locked dependency). That dist file is a self-contained classic script assigning globalThis mermaid - no imports, no runtime fetches - so no ES-module/CORS workaround is needed. The view shares the KaTeX skeleton: network stack blocked, file/content access off, asset URLs through the file:///android_asset/mermaid/ base URL (OfflineMathWebViewClient generalized to OfflineMarkdownWebViewClient, no behavior change). Behavior mirrors the web component line by line: initialize with startOnLoad:false, securityLevel:'strict', suppressErrorRendering:true and the web's dark/light themeVariables palettes verbatim (theme flips re-render); the validate-then-render gate parses with suppressErrors first and never calls render() after a failed validation, re-parsing once without suppression only to surface a human-readable reason; svg element ids are per-instance (`mermaid-<n>`, web useId) so the scoped styles mermaid emits cannot collide across diagrams in one message; wide diagrams scroll horizontally inside the document while the svg is centered at max-width:100% and height:auto. The code header (language chip + copy) stays above the body like the web's retained DefaultCodeBlock header; until the diagram renders - and on failure - the raw source is shown below it (the block's source exit), with the failure notice appearing only for actual errors so the loading window never flashes a render-error line. mermaidFallbackUi maps state to those surfaces as a pure function. 15 JVM tests cover dispatch, offline-ness, palettes, security flags, the validation gate structure, id uniqueness, generation filtering, script-breakout safety and the fallback mapping; the driver's runtime semantics were additionally validated against the real bundle in jsdom (flowchart/sequence render, invalid source surfaces the parse reason, hostile labels are rejected/sanitized under strict). Co-Authored-By: Claude Code <noreply@anthropic.com> via [HAPI](https://hapi.run) Co-Authored-By: HAPI <noreply@hapi.run> * fix(android): convert CSS pixel heights correctly in the mermaid WebView Both height reporters feeding MermaidDiagramView — the document bridge's report() (documentElement/body scrollHeight) and OfflineMarkdownWebViewClient's onPageFinished re-measure — return CSS pixels, but the rendered branch ran the value through with(density) { heightPx.toDp() }, which treats it as Android physical pixels and divides by display density: on a 3x display a diagram measuring 180 CSS px received a 60dp box and was clipped into an unnecessarily scrolling WebView. Fix it exactly like the KaTeX base now does: the document pins `width=device-width, initial-scale=1` (making 1 CSS px = 1 dp at the default WebView scale an explicit property of the document rather than a WebView default), and the reported value crosses the boundary through the shared density-independent cssPxToDp helper — extracted as the pure mermaidRenderedHeightDp so the composable no longer holds any density conversion at all, keeping both markdown renderers on one boundary contract. (The KaTeXMathBlock repeat of the same bug is fixed by the rebased base; MathWebView.kt arrives converted already.) JVM tests: the viewport pin is asserted against the built document (demonstrated failing on the pre-fix template) and the conversion math is driven through the pure helper — 180 CSS px stays 180dp because the boundary carries no density parameter, mirroring the base's MathDocumentTest contract. Co-Authored-By: Claude Code <noreply@anthropic.com> * feat(android): generalize generated media blocks to video/audio/file The hub's /generated-images/:id endpoint already serves any mime type (video, audio, arbitrary files arrive as the same generated-image wire block with a mimeType), but the Android chat rendered every block as an image. This ports the web GeneratedImageCard generalization so the card routes by display intent, exactly like web generatedInlineMedia.ts: - mimeType routing is the web's literal prefix check: video/… gets a player, audio/… an audio player, image/… or null keep the existing image treatment (unchanged AsyncImage thumbnail + tap-to-zoom dialog), everything else is a file card (no separate "unsupported" class — unknown mimes ARE the file UI, matching web G3/G8). - non-image media never downloads on its own: "Load video" / "Load audio" / "Prepare download" gate buttons (web's explicit-intent rule G4; English labels verbatim from the web card), pulsing skeleton while fetching (G7), and the shared "{label} is unavailable. {error}" error line (G8). The wire carries no tiny/poster fields, so video renders bare — nothing to special-case. - every kind now sits in the web's G14 card shell with the localized "{label}: {fileName}" hint header; images additionally surface the error line on Coil failure instead of a silent empty hole. - downloads go through the same authed OkHttp client Coil uses (JWT + silent 401 re-auth + ETag disk cache), staged into a temp file and atomically renamed under cache/generated-media/; a completed file short-circuits later fetches (immutable, ETag-stable content) and non-2xx maps to "HTTP <code>" exactly like the web's ApiError. - video plays in-app via framework VideoView + MediaController (no new media dependencies: Media3 is not on the classpath and pulling it in for this card is not worth the MB); audio is a compact play/pause + scrub + m:ss row over MediaPlayer (web's native <audio> chrome degraded: volume/speed omitted, no waveform — web has none); files get Open (ACTION_VIEW over the existing FileProvider, new generated-media cache path) and Save (ACTION_CREATE_DOCUMENT picker) outlets in place of the browser download link. - ChatMedia gains a nullable fetchGeneratedMedia field (default null: previews/tests degrade to the unavailable line, so stack-only call sites merge unchanged). 17 new strings added to en/ru/zh-CN in one commit, mirroring en key order (KEYSYNC clean; zh count equal — its pre-existing regional order drift is untouched). 10 JVM tests cover routing (including case sensitivity and null-mime), mount-gating, label/action mapping, cache file name sanitizing (path traversal), the MockWebServer round-trip with status mapping + cached-file short-circuit, and the time formatter. Gate: :app:testDebugUnitTest :core:data:test :core:protocol:test :app:lintDebug :app:compileDebugAndroidTestKotlin BUILD SUCCESSFUL. Co-Authored-By: Claude Code <noreply@anthropic.com> via [HAPI](https://hapi.run) Co-Authored-By: HAPI <noreply@hapi.run> * fix(android): refine generic file MIME types from the filename The file card passed the wire mimeType straight through to ACTION_VIEW, substituting application/octet-stream only when the wire type was null (r4370795783). Passing octet-stream explicitly defeats Android's viewer matching: a PDF viewer registered for application/pdf is not matched by an intent whose type says application/octet-stream, so PDFs (and every other ordinary file whose hub type was octet-stream or missing) opened with no eligible activity. resolveFileMimeType now passes a specific wire type through untouched, but when the wire type is missing, empty, or octet-stream (compared case-insensitively) it re-derives the type from the original filename's extension through an injected resolver — android.webkit.MimeTypeMap at the call site, following the established AttachmentPreparer / AndroidShareIntake pattern (lowercased extension, slash/wildcard sanity check on the resolved value). Unknown extensions keep the generic octet-stream fallback. The resolved type feeds both outlets: the Open ACTION_VIEW intent and the Save ACTION_CREATE_DOCUMENT contract. The resolver is a required parameter rather than a default, which keeps GeneratedMedia.kt free of android.* imports and forces the seam: 3 JVM tests drive it with a fake table (pdf->application/pdf via extension for null/octet-stream/empty wire types; unknown extension and no extension -> octet-stream fallback; specific wire types pass through untouched even when the extension would map elsewhere). All three fail on the pre-fix code by construction — resolveFileMimeType did not exist and the null-wire PDF case asserted the old octet-stream handoff. Gate: :app:testDebugUnitTest --tests GeneratedMediaTest BUILD SUCCESSFUL (13 tests, 0 failures). Co-Authored-By: Claude Code <noreply@anthropic.com> * fix(android): bound generated media cache names by UTF-8 bytes sanitizedGeneratedMediaFileName capped the stem at 96 UTF-16 chars and relied on that alone (r4370795787). Android filename limits are byte limits (NAME_MAX 255): 80 hanzi + ".pdf" is a 122-char / 281-byte component once the UUID imageId prefix is attached (verified against the pre-fix logic: 36-byte id + 1 + 240 + 4 = 281 bytes), so long Chinese names failed the atomic cache rename outright. The complete component — id prefix + stem + extension — is now assembled under a 255-byte UTF-8 budget, with pathological ids and extensions capped at 64/32 bytes so the stem always keeps room. The stem keeps its historical 96-char cap (ASCII names keep their exact prior shape), bytes beyond the budget are cut from the stem only, on code-point boundaries, and the extension always survives intact. The char-level whitelist already drops surrogate halves (a lone half is never a letter), so every surviving char is one complete BMP code point; the truncation helpers are additionally pair-safe for any future code-point-aware whitelist. JVM tests: the reviewer's UUID + 80-hanzi ".pdf" case lands at <= 255 bytes with the extension and id prefix preserved (red on the pre-fix logic, which produced exactly 281); ".tar.gz"-style extensions survive stem truncation; a battery of multibyte names re-decode byte-identically under the cap; ASCII names keep their historical shapes ("i-" + 96 a's, "img-1-media", "id-x.bin", "i-截图01.png"). Gate: :app:testDebugUnitTest --tests GeneratedMediaTest BUILD SUCCESSFUL (16 tests, 0 failures). Co-Authored-By: Claude Code <noreply@anthropic.com> * test(android): cover audio playback wiring and the file outlets' types The play/pause/seek decisions and the MediaPlayer lifecycle lived inline in the audio composable, which made them unreachable from JVM tests (no emulator here; androidTest only compile-checks). Extract them into GeneratedAudioSession over a GeneratedAudioPlayer seam — the app side adapts android.media.MediaPlayer, tests pass a scriptable fake — and the card now collects the session's flows, with DisposableEffect.onDispose forwarding to session.release(). JVM coverage through the fake: - prepare wires the data source + prepareAsync and reports duration on the prepared callback; pre-preparation toggles/seeks stay inert; - the toggle maps to start/pause (turbine on the playing flow) and natural completion parks the row; - seekToFraction lands fraction*duration and clamps to [0, duration], reading the position back afterwards; - pollPosition only reads while playing (the 250 ms LaunchedEffect); - release — the dispose hook — releases the framework player exactly once across double-dispose, stops playback, and every later control call is inert (a released MediaPlayer throws); - player errors and a failing data source both map to the failed line and the row goes dead. The file card's Open/Save construction moves behind generatedFileOutlets so a JVM test pins that ACTION_VIEW and ACTION_CREATE_DOCUMENT both carry the *resolved* MIME type (inferred pdf, generic fallback, specific wire passthrough) plus the suggested save name — android.content.Intent is not on the unit-test classpath. A MockWebServer fetch with the reviewer's 80-hanzi + UUID name proves the byte-bounded component really round-trips the local filesystem: under 255 bytes, extension kept, no staging litter beside the final file. These tests fail on the pre-change code by construction: the session, seam, and outlets did not exist and the logic was not reachable from the JVM. Explicit loading (loadsOnMount gate), authenticated caching (hubGraph.session.imageClient: JWT + silent 401 re-auth + ETag cache), and download cleanup (staged temp deleted on failure, atomic rename on success) are untouched — only their test surface grew. Gate: :app:testDebugUnitTest --tests GeneratedMediaTest BUILD SUCCESSFUL (24 tests, 0 failures). Co-Authored-By: Claude Code <noreply@anthropic.com> --------- Co-authored-by: Claude Code <noreply@anthropic.com> Co-authored-by: HAPI <noreply@hapi.run>
This commit is contained in:
co-authored by
Claude Code
HAPI
parent
3208199f99
commit
be5f37b9bf
@@ -0,0 +1,635 @@
|
||||
package app.hapi.protocol.markdown
|
||||
|
||||
import org.commonmark.node.Code
|
||||
import org.commonmark.node.Document
|
||||
import org.commonmark.node.FencedCodeBlock
|
||||
import org.commonmark.node.HardLineBreak
|
||||
import org.commonmark.node.HtmlBlock
|
||||
import org.commonmark.node.HtmlInline
|
||||
import org.commonmark.node.Image
|
||||
import org.commonmark.node.IndentedCodeBlock
|
||||
import org.commonmark.node.Link
|
||||
import org.commonmark.node.LinkReferenceDefinition
|
||||
import org.commonmark.node.Node
|
||||
import org.commonmark.node.Paragraph
|
||||
import org.commonmark.node.SoftLineBreak
|
||||
import org.commonmark.node.Text
|
||||
|
||||
/**
|
||||
* Math extraction, port of the web pipeline so both renderers agree on chat
|
||||
* markdown (`web/src/lib/remark-latex-bracket-math.ts` + `remark-math`
|
||||
* `singleDollarTextMath: false` semantics):
|
||||
*
|
||||
* - `$$...$$` on its own lines -> [MathKind.DISPLAY] (micromark math flow);
|
||||
* mid-paragraph `$$...$$` -> [MathKind.INLINE] (math text), closing run must
|
||||
* be the same size as the opening run.
|
||||
* - Single `$...$` NEVER opens math (currency protection, GitHub parity).
|
||||
* - `\[...\]` -> [MathKind.DISPLAY], `\( ... \)` -> [MathKind.INLINE]; the
|
||||
* opener `\(`/`\[` may be escaped by an odd run of backslashes.
|
||||
* - Code spans, fenced/indented code, html and link/image/definition
|
||||
* destinations never trigger math, and an unclosed opener is left as text.
|
||||
*
|
||||
* Mechanism (same as the web plugin): parse the repaired source once to learn
|
||||
* which source ranges are protected, replace every formula with private-use
|
||||
* placeholder tokens that keep the line shape (so the re-parse does not split
|
||||
* the paragraph), re-parse the safe source, then restore the tokens as
|
||||
* [MathBlockNode]/[MathInlineNode] and hoist display math out of paragraphs.
|
||||
*
|
||||
* Pure JVM on purpose: unit-tested in `:core:protocol` (MarkdownMathTest
|
||||
* mirrors the web remark-latex-bracket-math/markdown-text test names).
|
||||
*/
|
||||
internal object MarkdownMath {
|
||||
|
||||
/** Cheap pre-check: sources without any of these substrings skip the whole two-parse dance. */
|
||||
val MATH_HINT: Regex = Regex("""\$\$|\\\[|\\\(""")
|
||||
|
||||
private const val PLACEHOLDER_BASE = 0xE000
|
||||
private const val PLACEHOLDER_END = 0xF8FF
|
||||
|
||||
private val LIST_MARKER: Regex = Regex("""^(?:[-+*]|\d{1,9}[.)])[ \t]+""")
|
||||
|
||||
// ── Result types ─────────────────────────────────────────────────────────
|
||||
|
||||
/** One extracted formula plus its source span and its container prefix. */
|
||||
internal class MathMatch(
|
||||
val kind: MathKind,
|
||||
val value: String,
|
||||
val start: Int,
|
||||
val end: Int,
|
||||
val prefix: String,
|
||||
) {
|
||||
internal var token: Char = ' '
|
||||
}
|
||||
|
||||
/** The token-replaced source plus every extracted formula. */
|
||||
internal class PreparedMath(val safeSource: String, val matches: List<MathMatch>)
|
||||
|
||||
/**
|
||||
* Full pipeline for sources that pass [MATH_HINT]: mask-protected parse,
|
||||
* extraction, placeholder re-parse and restore. Returns the final document.
|
||||
*/
|
||||
fun parseWithMath(
|
||||
source: String,
|
||||
spannedParse: (String) -> Node,
|
||||
plainParse: (String) -> Node,
|
||||
): Document {
|
||||
val maskTree = spannedParse(source)
|
||||
val prepared = extract(source, maskTree as Document) ?: return maskTree as Document
|
||||
val document = plainParse(prepared.safeSource) as Document
|
||||
restore(document, prepared)
|
||||
return document
|
||||
}
|
||||
|
||||
// ── Extraction ───────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Find every formula in [source], using [tree] (parsed with source spans
|
||||
* enabled) to learn protected ranges. Returns null when there is no math,
|
||||
* in which case the caller keeps the original tree as-is.
|
||||
*/
|
||||
fun extract(source: String, tree: Document): PreparedMath? {
|
||||
val mask = BooleanArray(source.length)
|
||||
markProtectedRanges(source, tree, mask)
|
||||
|
||||
val matches = mutableListOf<MathMatch>()
|
||||
matches += findDollarMath(source, mask)
|
||||
// Existing $$ math is opaque to the bracket scan (web SOURCE_OPAQUE
|
||||
// contains math/inlineMath: `$$\verb|\(x\)|$$` keeps its inner `\(`).
|
||||
for (match in matches) markRange(mask, match.start, match.end)
|
||||
matches += findBracketMathMatches(source, mask)
|
||||
if (matches.isEmpty()) return null
|
||||
|
||||
matches.sortBy { it.start }
|
||||
val out = StringBuilder(source.length)
|
||||
var cursor = 0
|
||||
var nextToken = PLACEHOLDER_BASE
|
||||
for (match in matches) {
|
||||
val token = findUnusedPlaceholderToken(source, nextToken)
|
||||
nextToken = token.code + 1
|
||||
match.token = token
|
||||
out.append(source, cursor, match.start)
|
||||
out.append(makeSourcePlaceholder(source.substring(match.start, match.end), match.prefix, token))
|
||||
cursor = match.end
|
||||
}
|
||||
out.append(source, cursor, source.length)
|
||||
return PreparedMath(out.toString(), matches.toList())
|
||||
}
|
||||
|
||||
// ── Dollar math (remark-math `singleDollarTextMath: false` semantics) ────
|
||||
|
||||
/**
|
||||
* `$$` runs: flow math when the run opens a line (after a container
|
||||
* prefix or <= 3 spaces) with only whitespace after it, closing on a later
|
||||
* line inside the same container that is just a `$$` run of >= the opening
|
||||
* size; otherwise text math inside a paragraph, closing on an equal-sized
|
||||
* run with no blank line or protected range in between. Single `$` never
|
||||
* opens math.
|
||||
*/
|
||||
private fun findDollarMath(source: String, mask: BooleanArray): List<MathMatch> {
|
||||
val matches = mutableListOf<MathMatch>()
|
||||
var cursor = 0
|
||||
while (cursor < source.length) {
|
||||
val p = source.indexOf('$', cursor)
|
||||
if (p < 0) break
|
||||
var size = 0
|
||||
while (p + size < source.length && source[p + size] == '$') size++
|
||||
if (size < 2 || isProtected(mask, p, size) || !isUnescapedDelimiter(source, p)) {
|
||||
cursor = p + 1
|
||||
continue
|
||||
}
|
||||
|
||||
val lineStart = source.lastIndexOf('\n', p - 1) + 1
|
||||
val lineEnd = source.indexOf('\n', p).let { if (it < 0) source.length else it }
|
||||
val container = getMarkdownContainerPrefix(source, p)
|
||||
val lineIndent = source.substring(lineStart, p).takeWhile { it == ' ' || it == '\t' }.length
|
||||
val restOfLineBlank = source.substring(p + size, lineEnd).isBlank()
|
||||
val atFlowPosition = p - lineStart == container.prefix.length &&
|
||||
(container.recognized || lineIndent <= 3)
|
||||
|
||||
if (atFlowPosition && restOfLineBlank) {
|
||||
val close = findFlowClose(source, mask, size, container.prefix, container.recognized, lineEnd)
|
||||
if (close != null) {
|
||||
val value = stripLinePrefix(source.substring(lineEnd + 1, close.start), container.prefix).trim()
|
||||
if (value.isNotEmpty()) {
|
||||
matches.add(MathMatch(MathKind.DISPLAY, value, p, close.end, container.prefix))
|
||||
cursor = close.end
|
||||
continue
|
||||
}
|
||||
}
|
||||
// Unclosed flow math falls through to the paragraph-text rules,
|
||||
// matching micromark: the opener line stays a paragraph.
|
||||
}
|
||||
|
||||
val textClose = findTextClose(source, mask, p, size)
|
||||
if (textClose != null) {
|
||||
val raw = source.substring(p + size, textClose)
|
||||
val value = trimMirroredWhitespace(raw)
|
||||
if (value.isNotEmpty()) {
|
||||
matches.add(MathMatch(MathKind.INLINE, value, p, textClose + size, ""))
|
||||
cursor = textClose + size
|
||||
continue
|
||||
}
|
||||
}
|
||||
cursor = p + size
|
||||
}
|
||||
return matches
|
||||
}
|
||||
|
||||
private class CloseSpan(val start: Int, val end: Int)
|
||||
|
||||
/**
|
||||
* Closing `$$` line for flow math: container prefix / <= 3 spaces, then a `$` run >= [size].
|
||||
*
|
||||
* Bounded to the opening container (micromark math flow is `concrete` and
|
||||
* ends with its container): when [containerRecognized] the closer must
|
||||
* come from a line that still carries [prefix] — a quoted opener never
|
||||
* pairs with a later top-level or sibling `$$`. A blank line closes a
|
||||
* blockquote (a following `> $$` opens a NEW quote, not a closer), while a
|
||||
* list item may resume after blank lines (loose lists), so blanks are
|
||||
* skipped and the next content line decides. An unrecognized prefix is
|
||||
* plain <= 3 space indentation, not a container: no bounding, blank lines
|
||||
* are math content (web parity with `$$\nx\n\ny\n$$`).
|
||||
*/
|
||||
private fun findFlowClose(
|
||||
source: String,
|
||||
mask: BooleanArray,
|
||||
size: Int,
|
||||
prefix: String,
|
||||
containerRecognized: Boolean,
|
||||
openLineEnd: Int,
|
||||
): CloseSpan? {
|
||||
val marker = prefix.trimEnd()
|
||||
val quoted = marker.isNotEmpty() && marker.last() == '>'
|
||||
var lineStart = openLineEnd + 1
|
||||
while (lineStart < source.length) {
|
||||
val lineEnd = source.indexOf('\n', lineStart).let { if (it < 0) source.length else it }
|
||||
val line = source.substring(lineStart, lineEnd)
|
||||
if (containerRecognized && !line.startsWith(prefix)) {
|
||||
when {
|
||||
// A `>`-only line is a blank line inside the quote: the
|
||||
// container survives it ("> $$\n> x\n>\n> $$" closes).
|
||||
quoted && line.trimEnd(' ', '\t') == marker -> Unit
|
||||
// Blank line: the blockquote dies here, a list item may
|
||||
// resume after it — keep scanning for the list case only.
|
||||
line.isBlank() && !quoted -> Unit
|
||||
else -> return null
|
||||
}
|
||||
lineStart = lineEnd + 1
|
||||
continue
|
||||
}
|
||||
var contentStart = lineStart
|
||||
var content = line
|
||||
if (prefix.isNotEmpty() && content.startsWith(prefix)) {
|
||||
contentStart += prefix.length
|
||||
content = content.substring(prefix.length)
|
||||
}
|
||||
var indent = 0
|
||||
while (indent < content.length && indent < 3 && (content[indent] == ' ' || content[indent] == '\t')) indent++
|
||||
val rest = content.substring(indent)
|
||||
var run = 0
|
||||
while (run < rest.length && rest[run] == '$') run++
|
||||
if (run >= size && rest.substring(run).isBlank() && !isProtected(mask, contentStart + indent, run)) {
|
||||
val runStart = contentStart + indent
|
||||
return CloseSpan(runStart, runStart + run)
|
||||
}
|
||||
lineStart = lineEnd + 1
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
/**
|
||||
* Lines that would interrupt the opening paragraph (heading, blockquote,
|
||||
* fence, thematic break): micromark math-text cannot cross them, so the
|
||||
* search fails there. A blank line is handled inline below.
|
||||
*/
|
||||
private val PARAGRAPH_INTERRUPT: Regex = Regex("""^[ \t]{0,3}(?:#|>|```|~~~|[-*_][ \t]*[-*_][ \t]*[-*_])""")
|
||||
|
||||
/** Closing equal-sized `$$` run for text math: same paragraph, unescaped, nothing protected in between. */
|
||||
private fun findTextClose(source: String, mask: BooleanArray, openStart: Int, size: Int): Int? {
|
||||
var i = openStart + size
|
||||
while (i < source.length) {
|
||||
val c = source[i]
|
||||
if (c == '\n') {
|
||||
// A blank line ends the paragraph; text math cannot cross it.
|
||||
var j = i + 1
|
||||
while (j < source.length && (source[j] == ' ' || source[j] == '\t')) j++
|
||||
if (j >= source.length || source[j] == '\n') return null
|
||||
// Nor can another block construct that interrupts paragraphs.
|
||||
if (PARAGRAPH_INTERRUPT.containsMatchIn(source.substring(j).lineSequence().first())) return null
|
||||
i++
|
||||
} else if (c == '$') {
|
||||
var run = 0
|
||||
while (i + run < source.length && source[i + run] == '$') run++
|
||||
if (run == size && !isProtected(mask, i, run) && isUnescapedDelimiter(source, i)) {
|
||||
return if (containsProtectedRange(mask, openStart, i)) null else i
|
||||
}
|
||||
i += run
|
||||
} else {
|
||||
i++
|
||||
}
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
/** micromark math-text resolve: trimmed only when BOTH edges are whitespace. */
|
||||
private fun trimMirroredWhitespace(value: String): String {
|
||||
if (value.isEmpty()) return value
|
||||
val firstWs = value.first().isWhitespace()
|
||||
val lastWs = value.last().isWhitespace()
|
||||
return if (firstWs && lastWs) value.trim() else value
|
||||
}
|
||||
|
||||
// ── Bracket math (remark-latex-bracket-math port) ────────────────────────
|
||||
|
||||
private fun findBracketMathMatches(source: String, mask: BooleanArray): List<MathMatch> {
|
||||
val matches = mutableListOf<MathMatch>()
|
||||
var cursor = 0
|
||||
while (true) {
|
||||
val displayStart = findNextDelimiter(source, "\\[", cursor, mask)
|
||||
val inlineStart = findNextDelimiter(source, "\\(", cursor, mask)
|
||||
|
||||
val start: Int
|
||||
val kind: MathKind
|
||||
if (displayStart >= 0 && (inlineStart < 0 || displayStart < inlineStart)) {
|
||||
start = displayStart
|
||||
kind = MathKind.DISPLAY
|
||||
} else if (inlineStart >= 0) {
|
||||
start = inlineStart
|
||||
kind = MathKind.INLINE
|
||||
} else {
|
||||
break
|
||||
}
|
||||
|
||||
val closingDelimiter = if (kind == MathKind.DISPLAY) "\\]" else "\\)"
|
||||
val closingStart = findClosingDelimiter(source, closingDelimiter, start + 2, mask)
|
||||
if (closingStart < 0) {
|
||||
cursor = start + 2
|
||||
continue
|
||||
}
|
||||
|
||||
val prefix = getMarkdownContainerPrefix(source, start).prefix
|
||||
val value = stripLinePrefix(source.substring(start + 2, closingStart), prefix).trim()
|
||||
val end = closingStart + 2
|
||||
if (value.isNotEmpty()) matches.add(MathMatch(kind, value, start, end, prefix))
|
||||
cursor = end
|
||||
}
|
||||
return matches
|
||||
}
|
||||
|
||||
/** An odd run of backslashes before the delimiter escapes the final backslash. */
|
||||
private fun isUnescapedDelimiter(source: String, offset: Int): Boolean {
|
||||
var precedingBackslashes = 0
|
||||
var index = offset - 1
|
||||
while (index >= 0 && source[index] == '\\') {
|
||||
precedingBackslashes++
|
||||
index--
|
||||
}
|
||||
return precedingBackslashes % 2 == 0
|
||||
}
|
||||
|
||||
private fun findNextDelimiter(source: String, delimiter: String, from: Int, mask: BooleanArray): Int {
|
||||
var offset = source.indexOf(delimiter, from)
|
||||
while (offset >= 0 && (isProtected(mask, offset, delimiter.length) || !isUnescapedDelimiter(source, offset))) {
|
||||
offset = source.indexOf(delimiter, offset + 1)
|
||||
}
|
||||
return offset
|
||||
}
|
||||
|
||||
private fun findClosingDelimiter(source: String, delimiter: String, from: Int, mask: BooleanArray): Int {
|
||||
val offset = findNextDelimiter(source, delimiter, from, mask)
|
||||
return if (offset >= 0 && !containsProtectedRange(mask, from, offset)) offset else -1
|
||||
}
|
||||
|
||||
// ── Container prefixes ───────────────────────────────────────────────────
|
||||
|
||||
internal class ContainerPrefix(val prefix: String, val recognized: Boolean)
|
||||
|
||||
/**
|
||||
* Container prefix (blockquote markers, list markers, <= 3 spaces) that
|
||||
* precedes [offset] on its line; list markers normalize to spaces
|
||||
* (continuation indent). Returns the raw leading text when no container is
|
||||
* recognized and the delimiter is preceded by non-blank prose (web parity).
|
||||
*/
|
||||
private fun getMarkdownContainerPrefix(source: String, offset: Int): ContainerPrefix {
|
||||
val lineStart = source.lastIndexOf('\n', offset - 1) + 1
|
||||
val beforeDelimiter = source.substring(lineStart, offset)
|
||||
var cursor = 0
|
||||
val continuation = StringBuilder()
|
||||
var recognizedContainer = false
|
||||
|
||||
while (cursor < beforeDelimiter.length) {
|
||||
val containerStart = cursor
|
||||
while (cursor < beforeDelimiter.length &&
|
||||
(beforeDelimiter[cursor] == ' ' || beforeDelimiter[cursor] == '\t')
|
||||
) cursor++
|
||||
if (cursor - containerStart > 3) {
|
||||
cursor = containerStart
|
||||
break
|
||||
}
|
||||
|
||||
if (cursor < beforeDelimiter.length && beforeDelimiter[cursor] == '>') {
|
||||
cursor++
|
||||
if (cursor < beforeDelimiter.length &&
|
||||
(beforeDelimiter[cursor] == ' ' || beforeDelimiter[cursor] == '\t')
|
||||
) cursor++
|
||||
continuation.append(beforeDelimiter, containerStart, cursor)
|
||||
recognizedContainer = true
|
||||
continue
|
||||
}
|
||||
|
||||
val listMarker = LIST_MARKER.find(beforeDelimiter.substring(cursor))
|
||||
if (listMarker != null) {
|
||||
repeat(cursor - containerStart + listMarker.value.length) { continuation.append(' ') }
|
||||
cursor += listMarker.value.length
|
||||
recognizedContainer = true
|
||||
continue
|
||||
}
|
||||
|
||||
cursor = containerStart
|
||||
break
|
||||
}
|
||||
|
||||
if (!recognizedContainer && beforeDelimiter.isNotBlank()) return ContainerPrefix("", false)
|
||||
val prefix = continuation.toString().ifEmpty { beforeDelimiter }
|
||||
return ContainerPrefix(prefix, recognizedContainer)
|
||||
}
|
||||
|
||||
/** Remove [prefix] from the start of every line in [value], including the first. */
|
||||
private fun stripLinePrefix(value: String, prefix: String): String {
|
||||
if (prefix.isEmpty()) return value
|
||||
return value.replace(Regex("(^|\\n)" + Regex.escape(prefix)), "$1")
|
||||
}
|
||||
|
||||
// ── Placeholder construction ─────────────────────────────────────────────
|
||||
|
||||
private fun findUnusedPlaceholderToken(source: String, start: Int): Char {
|
||||
for (code in start..PLACEHOLDER_END) {
|
||||
if (!source.contains(code.toChar())) return code.toChar()
|
||||
}
|
||||
throw IllegalStateException("Unable to allocate a LaTeX placeholder token")
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace every line of the raw span with the placeholder token, keeping
|
||||
* the line shape (newlines, container prefix) so the re-parse keeps the
|
||||
* formula inside its original block; an empty line takes one token so the
|
||||
* paragraph is not split (web `makeSourcePlaceholder`).
|
||||
*/
|
||||
private fun makeSourcePlaceholder(raw: String, prefix: String, token: Char): String {
|
||||
val out = StringBuilder(raw.length)
|
||||
val lines = raw.split('\n')
|
||||
for (index in lines.indices) {
|
||||
if (index > 0) out.append('\n')
|
||||
val line = lines[index]
|
||||
if (index == 0) {
|
||||
repeat(maxOf(1, line.length)) { out.append(token) }
|
||||
} else {
|
||||
val kept = if (prefix.isNotEmpty() && line.startsWith(prefix)) prefix else ""
|
||||
out.append(kept)
|
||||
repeat(maxOf(1, line.length - kept.length)) { out.append(token) }
|
||||
}
|
||||
}
|
||||
return out.toString()
|
||||
}
|
||||
|
||||
// ── Protected ranges (mask) ──────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Ranges the math scan must never touch, learned from a parse with
|
||||
* `IncludeSourceSpans.BLOCKS_AND_INLINES`: code blocks/html blocks/
|
||||
* definitions consume whole lines; inline code, inline html and links
|
||||
* (incl. images and autolinks, which the autolink extension emits as
|
||||
* [Link]s) protect their exact span. Web parity note: the web plugin only
|
||||
* protects link DESTINATIONS (math inside `[label]` renders); here the
|
||||
* whole link is protected, which cannot corrupt anything and at most
|
||||
* leaves label math as literal text.
|
||||
*/
|
||||
private fun markProtectedRanges(source: String, tree: Document, mask: BooleanArray) {
|
||||
val lineStarts = lineStartOffsets(source)
|
||||
val stack = ArrayDeque<Node>()
|
||||
stack.addLast(tree)
|
||||
while (stack.isNotEmpty()) {
|
||||
val node = stack.removeLast()
|
||||
when (node) {
|
||||
is FencedCodeBlock, is IndentedCodeBlock, is HtmlBlock, is LinkReferenceDefinition -> {
|
||||
for (span in node.sourceSpans) protectFullLine(mask, lineStarts, source.length, span.lineIndex)
|
||||
}
|
||||
is Code, is HtmlInline, is Link, is Image -> {
|
||||
for (span in node.sourceSpans) {
|
||||
if (span.lineIndex < lineStarts.size) {
|
||||
val start = lineStarts[span.lineIndex] + span.columnIndex
|
||||
markRange(mask, start, start + span.length)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
var child = node.firstChild
|
||||
while (child != null) {
|
||||
stack.addLast(child)
|
||||
child = child.next
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private fun lineStartOffsets(source: String): IntArray {
|
||||
val starts = ArrayList<Int>()
|
||||
starts.add(0)
|
||||
for (index in source.indices) {
|
||||
if (source[index] == '\n') starts.add(index + 1)
|
||||
}
|
||||
return starts.toIntArray()
|
||||
}
|
||||
|
||||
private fun protectFullLine(mask: BooleanArray, lineStarts: IntArray, sourceLength: Int, lineIndex: Int) {
|
||||
if (lineIndex >= lineStarts.size) return
|
||||
val start = lineStarts[lineIndex]
|
||||
val end = if (lineIndex + 1 < lineStarts.size) lineStarts[lineIndex + 1] - 1 else sourceLength
|
||||
markRange(mask, start, end)
|
||||
}
|
||||
|
||||
private fun markRange(mask: BooleanArray, start: Int, end: Int) {
|
||||
for (index in maxOf(0, start) until minOf(mask.size, end)) mask[index] = true
|
||||
}
|
||||
|
||||
private fun isProtected(mask: BooleanArray, start: Int, length: Int): Boolean {
|
||||
for (index in start until minOf(mask.size, start + length)) {
|
||||
if (mask[index]) return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
private fun containsProtectedRange(mask: BooleanArray, start: Int, end: Int): Boolean {
|
||||
for (index in maxOf(0, start) until minOf(mask.size, end)) {
|
||||
if (mask[index]) return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ── Restore ──────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Turn placeholder tokens in the re-parsed [document] into math nodes:
|
||||
* split text node runs into [Text]/[MathInlineNode]/[MathBlockNode]
|
||||
* (fragments of one formula across soft line breaks collapse to a single
|
||||
* node), then hoist display math out of paragraphs (web
|
||||
* `splitParagraphAroundDisplayMath`), dropping whitespace-only leftovers.
|
||||
*/
|
||||
fun restore(document: Document, prepared: PreparedMath) {
|
||||
val byToken = HashMap<Char, MathMatch>()
|
||||
for (match in prepared.matches) byToken[match.token] = match
|
||||
val emitted = HashSet<Char>()
|
||||
|
||||
// Pass 1: split every text node that contains a placeholder run.
|
||||
val texts = ArrayList<Text>()
|
||||
val stack = ArrayDeque<Node>()
|
||||
stack.addLast(document)
|
||||
while (stack.isNotEmpty()) {
|
||||
val node = stack.removeLast()
|
||||
if (node is Text) texts.add(node)
|
||||
var child = node.firstChild
|
||||
while (child != null) {
|
||||
stack.addLast(child)
|
||||
child = child.next
|
||||
}
|
||||
}
|
||||
for (text in texts) {
|
||||
val literal = text.literal.orEmpty()
|
||||
var hasToken = false
|
||||
for (c in literal) {
|
||||
if (byToken.containsKey(c)) {
|
||||
hasToken = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if (!hasToken) continue
|
||||
|
||||
var cursor = 0
|
||||
var index = 0
|
||||
while (index < literal.length) {
|
||||
val match = byToken[literal[index]]
|
||||
if (match == null) {
|
||||
index++
|
||||
continue
|
||||
}
|
||||
var runEnd = index
|
||||
while (runEnd < literal.length && literal[runEnd] == literal[index]) runEnd++
|
||||
if (index > cursor) text.insertBefore(Text(literal.substring(cursor, index)))
|
||||
if (emitted.add(match.token)) {
|
||||
text.insertBefore(
|
||||
if (match.kind == MathKind.DISPLAY) MathBlockNode(match.value) else MathInlineNode(match.value),
|
||||
)
|
||||
}
|
||||
cursor = runEnd
|
||||
index = runEnd
|
||||
}
|
||||
if (cursor < literal.length) text.insertBefore(Text(literal.substring(cursor)))
|
||||
text.unlink()
|
||||
}
|
||||
|
||||
// Pass 2: hoist display math out of paragraphs.
|
||||
val paragraphs = ArrayList<Paragraph>()
|
||||
stack.addLast(document)
|
||||
while (stack.isNotEmpty()) {
|
||||
val node = stack.removeLast()
|
||||
if (node is Paragraph && node.firstChild?.next == null) {
|
||||
// Fast path: single-child paragraphs either hold one math node
|
||||
// (hoist) or nothing to do; still collected for uniform handling.
|
||||
}
|
||||
if (node is Paragraph) paragraphs.add(node)
|
||||
var child = node.firstChild
|
||||
while (child != null) {
|
||||
stack.addLast(child)
|
||||
child = child.next
|
||||
}
|
||||
}
|
||||
for (paragraph in paragraphs) {
|
||||
var hasDisplay = false
|
||||
var child = paragraph.firstChild
|
||||
while (child != null) {
|
||||
if (child is MathBlockNode) {
|
||||
hasDisplay = true
|
||||
break
|
||||
}
|
||||
child = child.next
|
||||
}
|
||||
if (!hasDisplay) continue
|
||||
|
||||
var group = ArrayList<Node>()
|
||||
val flush = {
|
||||
val hasContent = group.any { !isBlankInline(it) }
|
||||
if (hasContent) {
|
||||
val rest = Paragraph()
|
||||
for (inline in group) {
|
||||
inline.unlink()
|
||||
rest.appendChild(inline)
|
||||
}
|
||||
paragraph.insertBefore(rest)
|
||||
} else {
|
||||
for (inline in group) inline.unlink()
|
||||
}
|
||||
group = ArrayList()
|
||||
}
|
||||
var node = paragraph.firstChild
|
||||
while (node != null) {
|
||||
val next = node.next
|
||||
if (node is MathBlockNode) {
|
||||
flush()
|
||||
node.unlink()
|
||||
paragraph.insertBefore(node)
|
||||
} else {
|
||||
group.add(node)
|
||||
}
|
||||
node = next
|
||||
}
|
||||
flush()
|
||||
paragraph.unlink()
|
||||
}
|
||||
}
|
||||
|
||||
/** Whitespace-only text, soft and hard line breaks carry no paragraph content. */
|
||||
private fun isBlankInline(node: Node): Boolean = when (node) {
|
||||
is Text -> node.literal.orEmpty().isBlank()
|
||||
is SoftLineBreak, is HardLineBreak -> true
|
||||
else -> false
|
||||
}
|
||||
}
|
||||
+38
-8
@@ -11,6 +11,7 @@ import org.commonmark.node.HtmlBlock
|
||||
import org.commonmark.node.ListBlock
|
||||
import org.commonmark.node.Node
|
||||
import org.commonmark.node.ThematicBreak
|
||||
import org.commonmark.parser.IncludeSourceSpans
|
||||
import org.commonmark.parser.Parser
|
||||
|
||||
/**
|
||||
@@ -24,6 +25,9 @@ import org.commonmark.parser.Parser
|
||||
* - [stripCjkAutolinkArtifacts] <- `web/src/lib/remark-strip-cjk-autolink.ts`
|
||||
* - [detectFilePathLinks] / [matchWholeFilePath] / [rewriteExplicitLinkTarget]
|
||||
* <- `web/src/lib/remark-file-path-links.ts`
|
||||
* - [MarkdownMath] <- `web/src/lib/remark-latex-bracket-math.ts` +
|
||||
* `remark-math {singleDollarTextMath: false}`
|
||||
* (`parse` extracts `$$`/`\[...\]`/`\(...\)` math nodes)
|
||||
*
|
||||
* Pure JVM on purpose: everything here is unit-tested in `:core:protocol`.
|
||||
*/
|
||||
@@ -182,12 +186,12 @@ object MarkdownTransforms {
|
||||
builder.enabledBlockTypes(BLOCK_TYPES_WITHOUT_INDENTED_CODE)
|
||||
|
||||
/**
|
||||
* The canonical HAPI markdown parser: GFM tables + strikethrough (double
|
||||
* tilde only, matching web `remark-gfm {singleTilde: false}` -- a single
|
||||
* tilde is common in shell prompts) + bare-URL autolink, with indented code
|
||||
* blocks disabled. Thread-safe and reusable.
|
||||
* The canonical HAPI markdown parser builder: GFM tables + strikethrough
|
||||
* (double tilde only, matching web `remark-gfm {singleTilde: false}` -- a
|
||||
* single tilde is common in shell prompts) + bare-URL autolink, with
|
||||
* indented code blocks disabled. Thread-safe and reusable once built.
|
||||
*/
|
||||
fun newParser(): Parser = disableIndentedCode(
|
||||
fun newParserBuilder(): Parser.Builder = disableIndentedCode(
|
||||
Parser.builder().extensions(
|
||||
listOf(
|
||||
TablesExtension.create(),
|
||||
@@ -195,12 +199,38 @@ object MarkdownTransforms {
|
||||
AutolinkExtension.create(),
|
||||
)
|
||||
)
|
||||
).build()
|
||||
)
|
||||
|
||||
/** [newParserBuilder] built: the canonical parser. */
|
||||
fun newParser(): Parser = newParserBuilder().build()
|
||||
|
||||
private val sharedParser: Parser by lazy { newParser() }
|
||||
|
||||
/** Full source pipeline: [repairTables] then parse with [newParser] config. */
|
||||
fun parse(source: String): Node = sharedParser.parse(repairTables(source))
|
||||
/**
|
||||
* Same configuration plus source spans on blocks and inlines; used only by
|
||||
* [MarkdownMath] to learn which source ranges are protected (code, links,
|
||||
* definitions) before scanning for math delimiters.
|
||||
*/
|
||||
private val spannedParser: Parser by lazy {
|
||||
newParserBuilder().includeSourceSpans(IncludeSourceSpans.BLOCKS_AND_INLINES).build()
|
||||
}
|
||||
|
||||
/**
|
||||
* Full source pipeline: [repairTables] then parse with [newParser] config.
|
||||
* Sources that look like they may contain math (`$$`, `\[`, `\(`) go
|
||||
* through [MarkdownMath.parseWithMath], which extracts
|
||||
* [MathBlockNode]/[MathInlineNode] nodes; everything else takes the plain
|
||||
* single-parse path.
|
||||
*/
|
||||
fun parse(source: String): Node {
|
||||
val repaired = repairTables(source)
|
||||
if (!MarkdownMath.MATH_HINT.containsMatchIn(repaired)) return sharedParser.parse(repaired)
|
||||
return MarkdownMath.parseWithMath(
|
||||
repaired,
|
||||
spannedParse = spannedParser::parse,
|
||||
plainParse = sharedParser::parse,
|
||||
)
|
||||
}
|
||||
|
||||
// ── CJK autolink artifact strip (remark-strip-cjk-autolink port) ─────────
|
||||
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
package app.hapi.protocol.markdown
|
||||
|
||||
import org.commonmark.node.CustomBlock
|
||||
import org.commonmark.node.CustomNode
|
||||
|
||||
/** Whether a formula came from block-level or inline-level delimiters. */
|
||||
enum class MathKind {
|
||||
/** `$$...$$` on its own lines or `\[...\]` — web `math` node, `math-display`. */
|
||||
DISPLAY,
|
||||
|
||||
/** `\( ... \)` or `$$...$$` mid-paragraph — web `inlineMath` node, `math-inline`. */
|
||||
INLINE,
|
||||
}
|
||||
|
||||
/**
|
||||
* Block-level display math extracted from the source by [MarkdownMath].
|
||||
*
|
||||
* Mirrors the web `math` mdast node produced by `remark-latex-bracket-math`
|
||||
* (`\[...\]`) and `remark-math` flow math (`$$...$$` on their own lines): both
|
||||
* end up as `pre > code.language-math.math-display`, rendered by rehype-katex
|
||||
* with `displayMode: true`.
|
||||
*
|
||||
* Extends [CustomBlock] so it can be hoisted to a top-level block sibling
|
||||
* (web `splitParagraphAroundDisplayMath` parity); before hoisting it may
|
||||
* transiently sit in inline position, where the Compose walker degrades it to
|
||||
* the monospace inline fallback instead.
|
||||
*/
|
||||
class MathBlockNode(val latex: String) : CustomBlock() {
|
||||
override fun toString(): String = "MathBlockNode(${latex.replace("\n", "\\n")})"
|
||||
}
|
||||
|
||||
/**
|
||||
* Inline math extracted from `\( ... \)` or mid-paragraph `$$...$$`.
|
||||
*
|
||||
* Mirrors the web `inlineMath` mdast node (rendered by rehype-katex without
|
||||
* display mode). Produced by splitting [org.commonmark.node.Text] nodes that
|
||||
* contain a placeholder token, so it appears at any inline nesting depth
|
||||
* (emphasis, links labels excluded — those are protected at extraction time).
|
||||
*/
|
||||
class MathInlineNode(val latex: String) : CustomNode() {
|
||||
override fun toString(): String = "MathInlineNode(${latex.replace("\n", "\\n")})"
|
||||
}
|
||||
@@ -0,0 +1,798 @@
|
||||
package app.hapi.protocol.markdown
|
||||
|
||||
import org.commonmark.ext.gfm.tables.TableBlock
|
||||
import org.commonmark.node.BlockQuote
|
||||
import org.commonmark.node.Code
|
||||
import org.commonmark.node.Document
|
||||
import org.commonmark.node.FencedCodeBlock
|
||||
import org.commonmark.node.Image
|
||||
import org.commonmark.node.Link
|
||||
import org.commonmark.node.ListItem
|
||||
import org.commonmark.node.Node
|
||||
import org.commonmark.node.Paragraph
|
||||
import org.commonmark.node.StrongEmphasis
|
||||
import org.commonmark.node.Text
|
||||
import kotlin.test.Test
|
||||
import kotlin.test.assertEquals
|
||||
import kotlin.test.assertTrue
|
||||
|
||||
/**
|
||||
* Math extraction semantics, mirroring the web test names in
|
||||
* `web/src/lib/remark-latex-bracket-math.test.ts` and the currency suite of
|
||||
* `web/src/components/assistant-ui/markdown-text.test.ts` (remark-math with
|
||||
* `singleDollarTextMath: false`). Assertions are AST-level: which
|
||||
* [MathBlockNode]/[MathInlineNode] nodes exist, their latex values and block
|
||||
* placement, and that no private-use placeholder token ever leaks into the
|
||||
* final tree.
|
||||
*/
|
||||
class MarkdownMathTest {
|
||||
|
||||
private fun parse(markdown: String): Document = MarkdownTransforms.parse(markdown) as Document
|
||||
|
||||
private fun <T : Node> Node.collectOfType(type: Class<T>): List<T> {
|
||||
val out = ArrayList<T>()
|
||||
fun walk(node: Node) {
|
||||
var child = node.firstChild
|
||||
while (child != null) {
|
||||
if (type.isInstance(child)) out.add(type.cast(child))
|
||||
walk(child)
|
||||
child = child.next
|
||||
}
|
||||
}
|
||||
walk(this)
|
||||
return out
|
||||
}
|
||||
|
||||
private inline fun <reified T : Node> Node.collect(): List<T> = collectOfType(T::class.java)
|
||||
|
||||
private fun Node.allText(): String {
|
||||
val out = StringBuilder()
|
||||
fun walk(node: Node) {
|
||||
var child = node.firstChild
|
||||
while (child != null) {
|
||||
if (child is Text) out.append(child.literal)
|
||||
else walk(child)
|
||||
child = child.next
|
||||
}
|
||||
}
|
||||
walk(this)
|
||||
return out.toString()
|
||||
}
|
||||
|
||||
private fun Document.mathBlocks(): List<MathBlockNode> = collectOfType(MathBlockNode::class.java)
|
||||
|
||||
private fun Document.mathInlines(): List<MathInlineNode> = collectOfType(MathInlineNode::class.java)
|
||||
|
||||
private fun Document.topBlocks(): List<Node> {
|
||||
val blocks = ArrayList<Node>()
|
||||
var child = firstChild
|
||||
while (child != null) {
|
||||
blocks.add(child)
|
||||
child = child.next
|
||||
}
|
||||
return blocks
|
||||
}
|
||||
|
||||
/** Web parity: `expect(html).not.toContain('\uE000')`. */
|
||||
private fun assertNoPlaceholderTokens(document: Document) {
|
||||
for (text in document.collect<Text>()) {
|
||||
for (ch in text.literal.orEmpty()) {
|
||||
assertTrue(
|
||||
ch.code !in 0xE000..0xF8FF,
|
||||
"placeholder token leaked into text: \"${text.literal}\"",
|
||||
)
|
||||
}
|
||||
}
|
||||
for (code in document.collect<Code>()) {
|
||||
for (ch in code.literal.orEmpty()) {
|
||||
assertTrue(ch.code !in 0xE000..0xF8FF, "placeholder token leaked into code span")
|
||||
}
|
||||
}
|
||||
for (fence in document.collect<FencedCodeBlock>()) {
|
||||
for (ch in fence.literal.orEmpty()) {
|
||||
assertTrue(ch.code !in 0xE000..0xF8FF, "placeholder token leaked into fenced code")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ── Bracket math ─────────────────────────────────────────────────────────
|
||||
|
||||
@Test
|
||||
fun `renders display bracket math`() {
|
||||
val document = parse("""\[E = mc^2\]""")
|
||||
|
||||
val math = document.collect<MathBlockNode>()
|
||||
assertEquals(listOf("E = mc^2"), math.map { it.latex })
|
||||
assertEquals(0, document.collect<MathInlineNode>().size)
|
||||
assertEquals(0, document.collect<Paragraph>().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `renders parenthesized inline math`() {
|
||||
val document = parse("""Result: \(x^2 + y^2\).""")
|
||||
|
||||
val math = document.collect<MathInlineNode>()
|
||||
assertEquals(listOf("x^2 + y^2"), math.map { it.latex })
|
||||
assertEquals(0, document.collect<MathBlockNode>().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `renders multiline display math before optional hard-break processing`() {
|
||||
val document = parse(
|
||||
"""
|
||||
Before
|
||||
|
||||
\[
|
||||
E = mc^2
|
||||
\]
|
||||
|
||||
After
|
||||
""".trimIndent(),
|
||||
)
|
||||
|
||||
val math = document.collect<MathBlockNode>()
|
||||
assertEquals(listOf("E = mc^2"), math.map { it.latex })
|
||||
val paragraphs = document.collect<Paragraph>().map { it.allText() }
|
||||
assertEquals(listOf("Before", "After"), paragraphs)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `renders multiple inline formulas in one paragraph`() {
|
||||
val document = parse("""Compare \(a^2\) with \(b^2\).""")
|
||||
|
||||
assertEquals(listOf("a^2", "b^2"), document.collect<MathInlineNode>().map { it.latex })
|
||||
assertEquals(1, document.collect<Paragraph>().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `renders aligned display math with TeX line breaks and indentation`() {
|
||||
val document = parse(
|
||||
"""
|
||||
\[
|
||||
\begin{aligned}
|
||||
f(x) &= x^2+2x+1 \\
|
||||
&= (x+1)^2
|
||||
\end{aligned}
|
||||
\]
|
||||
""".trimIndent(),
|
||||
)
|
||||
|
||||
val math = document.collect<MathBlockNode>()
|
||||
assertEquals(1, math.size)
|
||||
assertEquals(
|
||||
"\\begin{aligned}\nf(x) &= x^2+2x+1 \\\\\n &= (x+1)^2\n\\end{aligned}",
|
||||
math.single().latex,
|
||||
)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `hoists display math out of surrounding prose`() {
|
||||
val document = parse("""Before \[x^2\] after""")
|
||||
|
||||
val math = document.collect<MathBlockNode>()
|
||||
assertEquals(listOf("x^2"), math.map { it.latex })
|
||||
val paragraphs = document.collect<Paragraph>().map { it.allText() }
|
||||
assertEquals(listOf("Before ", " after"), paragraphs)
|
||||
// The hoisted math must sit between the two prose paragraphs.
|
||||
val blocks = document.topBlocks()
|
||||
assertEquals(3, blocks.size)
|
||||
assertTrue(
|
||||
blocks[1] is MathBlockNode,
|
||||
"expected math between the paragraphs, got ${blocks.map { it.javaClass.simpleName }}",
|
||||
)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `preserves common LaTeX commands inside display math`() {
|
||||
val document = parse(
|
||||
"""
|
||||
\[
|
||||
\lim_{x\to 0}\frac{\sin x}{x}=1
|
||||
\]
|
||||
""".trimIndent(),
|
||||
)
|
||||
|
||||
assertEquals(
|
||||
listOf("\\lim_{x\\to 0}\\frac{\\sin x}{x}=1"),
|
||||
document.collect<MathBlockNode>().map { it.latex },
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `renders the multiline transfer-function formulas from the real session`() {
|
||||
val document = parse(
|
||||
"""
|
||||
\[
|
||||
\Phi(s)=
|
||||
\frac{\dfrac{K^*(s+4)}{s(s+2)(s+3)}}
|
||||
{1+\dfrac{K^*}{s(s+3)}}
|
||||
\]
|
||||
|
||||
\[
|
||||
\Phi(s)
|
||||
=
|
||||
\frac{K^*(s+4)}
|
||||
{(s+2)\left[s(s+3)+K^*\right]}
|
||||
\]
|
||||
|
||||
\[
|
||||
\Phi(s)=
|
||||
\frac{9(s+4)}
|
||||
{(s+2)(s^2+3s+9)}
|
||||
\]
|
||||
|
||||
\[
|
||||
\Phi(s)
|
||||
=
|
||||
\frac{9}{s^2+3s+9}
|
||||
\frac{s+4}{s+2}
|
||||
\]
|
||||
""".trimIndent(),
|
||||
)
|
||||
|
||||
val math = document.collect<MathBlockNode>()
|
||||
assertEquals(4, math.size)
|
||||
assertTrue(math[0].latex.contains("\\dfrac"), "first formula lost its nested fraction")
|
||||
assertTrue(math[1].latex.contains("\\left[s(s+3)+K^*\\right]"))
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `renders percent and text commands from the real session`() {
|
||||
val document = parse(
|
||||
"""
|
||||
标准二阶系统超调量:
|
||||
|
||||
\[
|
||||
\sigma\% =
|
||||
e^{-\frac{\pi\zeta}{\sqrt{1-\zeta^2}}}\times100\%
|
||||
\]
|
||||
|
||||
\[
|
||||
t_s\approx\frac{4}{0.5\times3}
|
||||
\approx2.67\text{ s}
|
||||
\]
|
||||
""".trimIndent(),
|
||||
)
|
||||
|
||||
val math = document.collect<MathBlockNode>()
|
||||
assertEquals(2, math.size)
|
||||
assertTrue(math[1].latex.contains("\\text{ s}"))
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `recurses through inline formatting while keeping the display node block-level`() {
|
||||
val document = parse("""**Before \(x^2\) after**""")
|
||||
|
||||
val emphasis = document.collect<StrongEmphasis>()
|
||||
assertEquals(1, emphasis.size)
|
||||
assertEquals(listOf("x^2"), emphasis.single().collect<MathInlineNode>().map { it.latex })
|
||||
assertEquals(0, document.collect<MathBlockNode>().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
// ── Literal passthrough ──────────────────────────────────────────────────
|
||||
|
||||
@Test
|
||||
fun `leaves inline code as text`() {
|
||||
val document = parse("""Use `\(x^2\)` here.""")
|
||||
|
||||
assertEquals(0, document.collect<MathInlineNode>().size + document.collect<MathBlockNode>().size)
|
||||
assertEquals(listOf("\\(x^2\\)"), document.collect<Code>().map { it.literal })
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `leaves fenced code as text`() {
|
||||
val document = parse("```latex\n\\[x^2\\]\n```")
|
||||
|
||||
assertEquals(0, document.collect<MathInlineNode>().size + document.collect<MathBlockNode>().size)
|
||||
assertEquals(listOf("\\[x^2\\]\n"), document.collect<FencedCodeBlock>().map { it.literal })
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `leaves escaped delimiters as text`() {
|
||||
val document = parse("""Literal \\(x^2\\) and \\[y^2\\]""")
|
||||
|
||||
assertEquals(0, document.collect<MathInlineNode>().size + document.collect<MathBlockNode>().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `leaves an unclosed delimiter as text`() {
|
||||
val document = parse("""Literal \(x^2""")
|
||||
|
||||
assertEquals(0, document.collect<MathInlineNode>().size + document.collect<MathBlockNode>().size)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `does not let an unmatched opener consume a protected code block`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"""Literal \(""",
|
||||
"```latex",
|
||||
"""\[x^2\]""",
|
||||
"```",
|
||||
"",
|
||||
"""\(y^2\)""",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
assertEquals(listOf("\\[x^2\\]\n"), document.collect<FencedCodeBlock>().map { it.literal })
|
||||
assertEquals(listOf("y^2"), document.collect<MathInlineNode>().map { it.latex })
|
||||
assertTrue(document.allText().contains("Literal ("), "the lone opener must stay literal text")
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `preserves quoted fences and multiline code spans as literal Markdown`() {
|
||||
val quotedFence = parse(
|
||||
listOf(
|
||||
"> ```latex",
|
||||
"> \\[x^2\\]",
|
||||
"> ````",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
assertEquals(0, quotedFence.collect<MathInlineNode>().size + quotedFence.collect<MathBlockNode>().size)
|
||||
assertEquals(listOf("\\[x^2\\]\n"), quotedFence.collect<FencedCodeBlock>().map { it.literal })
|
||||
|
||||
val multilineCodeSpan = parse(
|
||||
listOf(
|
||||
"Use `before",
|
||||
"""\[x^2\]""",
|
||||
"after` here.",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
assertEquals(0, multilineCodeSpan.collect<MathInlineNode>().size + multilineCodeSpan.collect<MathBlockNode>().size)
|
||||
// CommonMark converts newlines inside a code span to spaces; the
|
||||
// delimiters just have to stay literal.
|
||||
assertEquals(listOf("before \\[x^2\\] after"), multilineCodeSpan.collect<Code>().map { it.literal })
|
||||
}
|
||||
|
||||
// ── Container prefixes ───────────────────────────────────────────────────
|
||||
|
||||
@Test
|
||||
fun `removes blockquote prefixes from multiline bracket math`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"> \\[",
|
||||
"> x = 1",
|
||||
"> \\]",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
val math = document.collect<MathBlockNode>()
|
||||
assertEquals(listOf("x = 1"), math.map { it.latex })
|
||||
assertEquals(1, document.collect<BlockQuote>().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `removes blockquote prefixes from dollar flow math`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"> $$",
|
||||
"> E = mc^2",
|
||||
"> F = ma",
|
||||
"> $$",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
val math = document.collect<MathBlockNode>()
|
||||
assertEquals(listOf("E = mc^2\nF = ma"), math.map { it.latex })
|
||||
assertEquals(1, document.collect<BlockQuote>().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
|
||||
val singleLine = parse(
|
||||
listOf(
|
||||
"> $$",
|
||||
"> E",
|
||||
"> $$",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
assertEquals(listOf("E"), singleLine.collect<MathBlockNode>().map { it.latex })
|
||||
assertNoPlaceholderTokens(singleLine)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `preserves blockquote prefixes when prose precedes multiline bracket math`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"> Result: \\[",
|
||||
"> x = 1",
|
||||
"> \\]",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
val math = document.collect<MathBlockNode>()
|
||||
assertEquals(listOf("x = 1"), math.map { it.latex })
|
||||
val quotes = document.collect<BlockQuote>()
|
||||
assertEquals(1, quotes.size)
|
||||
assertEquals(listOf("Result: "), quotes.single().collect<Paragraph>().map { it.allText() })
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `keeps mathematical greater-than signs in unquoted multiline bracket math`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"\\[",
|
||||
"x",
|
||||
"> 0",
|
||||
"\\]",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
assertEquals(listOf("x\n> 0"), document.collect<MathBlockNode>().map { it.latex })
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
// ── Lists and fences ─────────────────────────────────────────────────────
|
||||
|
||||
@Test
|
||||
fun `preserves list-contained fences and renders following bracket math`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"- ```latex",
|
||||
" \\[x^2\\]",
|
||||
" ````",
|
||||
"",
|
||||
"""\(y^2\)""",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
assertEquals(listOf("\\[x^2\\]\n"), document.collect<FencedCodeBlock>().map { it.literal })
|
||||
assertEquals(listOf("y^2"), document.collect<MathInlineNode>().map { it.latex })
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `recognizes ordered-list fence indentation before following bracket math`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"10. ```latex",
|
||||
" \\[x^2\\]",
|
||||
" ````",
|
||||
"",
|
||||
"""\(y^2\)""",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
assertEquals(listOf("\\[x^2\\]\n"), document.collect<FencedCodeBlock>().map { it.literal })
|
||||
assertEquals(listOf("y^2"), document.collect<MathInlineNode>().map { it.latex })
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `allows a list-relative closing fence indentation`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"- ```latex",
|
||||
" \\[x^2\\]",
|
||||
" ````",
|
||||
"",
|
||||
"""\(y^2\)""",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
assertEquals(listOf("\\[x^2\\]\n"), document.collect<FencedCodeBlock>().map { it.literal })
|
||||
assertEquals(listOf("y^2"), document.collect<MathInlineNode>().map { it.latex })
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `keeps multiline bracket math inside a list item`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"- \\[",
|
||||
" x = 1",
|
||||
" \\]",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
val math = document.collect<MathBlockNode>()
|
||||
assertEquals(listOf("x = 1"), math.map { it.latex })
|
||||
val items = document.collect<ListItem>()
|
||||
assertEquals(1, items.size)
|
||||
assertTrue(items.single().collect<MathBlockNode>().isNotEmpty(), "math must stay inside the list item")
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `normalizes nested list and blockquote prefixes in multiline bracket math`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"- > \\[",
|
||||
" > x = 1",
|
||||
" > \\]",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
val math = document.collect<MathBlockNode>()
|
||||
assertEquals(listOf("x = 1"), math.map { it.latex })
|
||||
assertEquals(1, document.collect<BlockQuote>().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `renders bracket math across an empty TeX line`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"\\[",
|
||||
"a",
|
||||
"",
|
||||
"+ b",
|
||||
"\\]",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
assertEquals(listOf("a\n\n+ b"), document.collect<MathBlockNode>().map { it.latex })
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
// ── Link / image / definition metadata ───────────────────────────────────
|
||||
|
||||
@Test
|
||||
fun `keeps escaped parentheses in link, image, and definition destinations`() {
|
||||
val link = parse("""[source](https://example.com/foo\(bar\).ts)""")
|
||||
assertEquals(listOf("https://example.com/foo(bar).ts"), link.collect<Link>().map { it.destination })
|
||||
assertEquals(0, link.collect<MathInlineNode>().size + link.collect<MathBlockNode>().size)
|
||||
|
||||
val image = parse(""".png)""")
|
||||
assertEquals(listOf("https://example.com/foo(bar).png"), image.collect<Image>().map { it.destination })
|
||||
assertEquals(0, image.collect<MathInlineNode>().size + image.collect<MathBlockNode>().size)
|
||||
|
||||
val definition = parse("[source][id]\n\n[id]: https://example.com/foo\\(bar\\).ts")
|
||||
val resolved = definition.collect<Link>().map { it.destination }
|
||||
assertEquals(listOf("https://example.com/foo(bar).ts"), resolved)
|
||||
assertEquals(0, definition.collect<MathInlineNode>().size + definition.collect<MathBlockNode>().size)
|
||||
assertNoPlaceholderTokens(definition)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `preserves image alt text and reference identifiers containing delimiters`() {
|
||||
val image = parse("""""")
|
||||
assertEquals(1, image.collect<Image>().size)
|
||||
assertEquals(0, image.collect<MathInlineNode>().size + image.collect<MathBlockNode>().size)
|
||||
assertNoPlaceholderTokens(image)
|
||||
|
||||
val reference = parse("[source][\\(id\\)]\n\n[\\(id\\)]: https://example.com/foo")
|
||||
assertEquals(1, reference.collect<Link>().size)
|
||||
assertEquals(0, reference.collect<MathInlineNode>().size + reference.collect<MathBlockNode>().size)
|
||||
assertNoPlaceholderTokens(reference)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `preserves reference-image alt text and implicit link identifiers`() {
|
||||
val imageReference = parse("![\\(x\\)][id]\n\n[id]: https://example.com/image.png")
|
||||
assertEquals(1, imageReference.collect<Image>().size)
|
||||
assertEquals(0, imageReference.collect<MathInlineNode>().size + imageReference.collect<MathBlockNode>().size)
|
||||
assertNoPlaceholderTokens(imageReference)
|
||||
|
||||
val collapsed = parse("[\\(x\\)][]\n\n[\\(x\\)]: https://example.com")
|
||||
assertEquals(1, collapsed.collect<Link>().size)
|
||||
assertNoPlaceholderTokens(collapsed)
|
||||
|
||||
val shortcut = parse("[\\(x\\)]\n\n[\\(x\\)]: https://example.com")
|
||||
assertEquals(1, shortcut.collect<Link>().size)
|
||||
assertNoPlaceholderTokens(shortcut)
|
||||
}
|
||||
|
||||
// ── Dollar math (remark-math `singleDollarTextMath: false`) ──────────────
|
||||
|
||||
@Test
|
||||
fun `does not render single-dollar currency amounts as math`() {
|
||||
val document = parse("The plan is \$200/mo and the bill is \$80 — total \$400 saved.")
|
||||
|
||||
assertEquals(0, document.collect<MathInlineNode>().size + document.collect<MathBlockNode>().size)
|
||||
assertTrue(document.allText().contains("\$200"))
|
||||
assertTrue(document.allText().contains("\$80"))
|
||||
assertTrue(document.allText().contains("\$400"))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `does not render the reported real-world prose as math`() {
|
||||
val document = parse("Cursor's UI quotes the ratio: at least \$400 of API usage on a \$200 plan. That's 2:1.")
|
||||
|
||||
assertEquals(0, document.collect<MathInlineNode>().size + document.collect<MathBlockNode>().size)
|
||||
assertTrue(document.allText().contains("\$400"))
|
||||
assertTrue(document.allText().contains("\$200"))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `still renders block math with dollar-dollar on its own lines`() {
|
||||
val document = parse("Before\n\n\$\$\nE = mc^2\n\$\$\n\nAfter")
|
||||
|
||||
assertEquals(listOf("E = mc^2"), document.collect<MathBlockNode>().map { it.latex })
|
||||
assertEquals(listOf("Before", "After"), document.collect<Paragraph>().map { it.allText() })
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `renders mid-paragraph dollar-dollar as inline math`() {
|
||||
val document = parse("Euler says \$\$e^{i\\pi} + 1 = 0\$\$ exactly.")
|
||||
|
||||
assertEquals(listOf("e^{i\\pi} + 1 = 0"), document.collect<MathInlineNode>().map { it.latex })
|
||||
assertEquals(0, document.collect<MathBlockNode>().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `requires a matching dollar run size for inline math`() {
|
||||
// A `$$$` opener needs an equal three-dollar closer; without one the
|
||||
// whole run stays literal (the inner `$$` is part of the opener).
|
||||
val document = parse("a \$\$\$x\$\$ b")
|
||||
|
||||
assertEquals(0, document.collect<MathInlineNode>().size + document.collect<MathBlockNode>().size)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `treats backslash-escaped dollars as literal`() {
|
||||
val document = parse("He said " + "\\$" + "\\$" + "x" + "\\$" + "\\$" + " loudly.")
|
||||
|
||||
assertEquals(0, document.collect<MathInlineNode>().size + document.collect<MathBlockNode>().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `does not close flow math through a fenced code block`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"\$\$",
|
||||
"```",
|
||||
"not a closer \$\$",
|
||||
"```",
|
||||
"\$\$",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
// The `$$` inside the fence is protected, so the real closer wins and
|
||||
// the fence lines become math content (math flow, like code fences,
|
||||
// is not interrupted by ```).
|
||||
assertEquals(
|
||||
listOf("```\nnot a closer \$\$\n```"),
|
||||
document.collect<MathBlockNode>().map { it.latex },
|
||||
)
|
||||
assertEquals(0, document.collect<FencedCodeBlock>().size)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `does not close quoted flow math from a later top-level opener`() {
|
||||
// Web parity (remark-math): math flow is concrete and ends with its
|
||||
// container, so the quoted `> $$` opener must not pair with the
|
||||
// top-level `$$` after the blockquote ended — ordinary prose stays
|
||||
// prose and the later equation keeps its own opener.
|
||||
val document = parse(
|
||||
listOf(
|
||||
"> \$\$",
|
||||
"> x",
|
||||
"",
|
||||
"Ordinary prose",
|
||||
"",
|
||||
"\$\$",
|
||||
"y",
|
||||
"\$\$",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
assertEquals(listOf("y"), document.mathBlocks().map { it.latex })
|
||||
val paragraphs = document.collect<Paragraph>().map { it.allText() }
|
||||
assertTrue("Ordinary prose" in paragraphs, "prose after the quote stays a prose paragraph")
|
||||
assertTrue(
|
||||
paragraphs.any { it.startsWith("\$\$") && it.endsWith("x") },
|
||||
"the unclosed quoted opener stays literal text (got: $paragraphs)",
|
||||
)
|
||||
assertEquals(1, document.collect<BlockQuote>().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `does not close list-contained flow math from a later top-level opener`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"- \$\$",
|
||||
" x",
|
||||
"",
|
||||
"Ordinary prose",
|
||||
"",
|
||||
"\$\$",
|
||||
"y",
|
||||
"\$\$",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
assertEquals(listOf("y"), document.mathBlocks().map { it.latex })
|
||||
val paragraphs = document.collect<Paragraph>().map { it.allText() }
|
||||
assertTrue("Ordinary prose" in paragraphs, "prose after the list stays a prose paragraph")
|
||||
assertTrue(
|
||||
paragraphs.any { it.startsWith("\$\$") && it.endsWith("x") },
|
||||
"the unclosed list opener stays literal text (got: $paragraphs)",
|
||||
)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `does not close quoted flow math across a blank line into a new quote`() {
|
||||
// A blank line closes the blockquote; the later `> $$` re-opens a NEW
|
||||
// quote and is not a closer for the first one (remark-math renders
|
||||
// two separate blocks here).
|
||||
val document = parse(
|
||||
listOf(
|
||||
"> \$\$",
|
||||
"> x",
|
||||
"",
|
||||
"> \$\$",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
assertEquals(0, document.mathBlocks().size)
|
||||
assertEquals(0, document.mathInlines().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `closes list-contained flow math on a prefixed closer line`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"- \$\$",
|
||||
" E = mc^2",
|
||||
" \$\$",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
// Bounded matching keeps in-container closers working.
|
||||
assertEquals(listOf("E = mc^2"), document.mathBlocks().map { it.latex })
|
||||
val items = document.collect<ListItem>()
|
||||
assertEquals(1, items.size)
|
||||
assertEquals(1, items.single().collect<MathBlockNode>().size)
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `closes list-contained flow math across a blank line in a loose item`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"- \$\$",
|
||||
" x",
|
||||
"",
|
||||
" \$\$",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
// A list item may resume after a blank line (loose list), so the
|
||||
// indented closer still pairs — only blockquotes die at blank lines.
|
||||
assertEquals(listOf("x"), document.mathBlocks().map { it.latex })
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun `preserves bracket-like TeX commands inside existing dollar math`() {
|
||||
val document = parse("\$\$\\verb|\\(x\\)|\$\$")
|
||||
|
||||
assertEquals(listOf("""\verb|\(x\)|"""), document.collect<MathInlineNode>().map { it.latex })
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
|
||||
// ── Interaction with table repair ────────────────────────────────────────
|
||||
|
||||
@Test
|
||||
fun `keeps bracket math after the existing table source repair`() {
|
||||
val document = parse(
|
||||
listOf(
|
||||
"| A | B | C |",
|
||||
"|---|---|",
|
||||
"| x | y | z |",
|
||||
"",
|
||||
"""\[E = mc^2\]""",
|
||||
).joinToString("\n"),
|
||||
)
|
||||
|
||||
assertEquals(1, document.collect<TableBlock>().size)
|
||||
assertEquals(listOf("E = mc^2"), document.collect<MathBlockNode>().map { it.latex })
|
||||
assertNoPlaceholderTokens(document)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user