Repo refactor: rename "website" to "docs"
This commit is contained in:
@@ -0,0 +1,241 @@
|
||||
#maturity/experimental
|
||||
|
||||
The `group by` and `having` clauses of [[Space Lua/Integrated Query]] support aggregate functions for grouped analysis, following SQL-style semantics.
|
||||
|
||||
After `group by`, each result row contains:
|
||||
|
||||
- `key`: the group key (a single value or, for multi-key grouping, a table)
|
||||
- `group`: a Lua table containing all items in that group
|
||||
|
||||
All aggregate functions (such as `count`, `sum`, `min`, `max`, `avg`, and custom aggregates) can be applied in `select` and `having` clauses. Aggregate expressions are available in both forms: with or without a variable binding in the `from` clause. The variable `_` always refers to the current item.
|
||||
|
||||
Field names used in `group by` are exposed as locals in `having`, `select`, and `order by`. Use `#group` to obtain the item count per group.
|
||||
|
||||
> **note** The `having` clause acts only on grouped output. For filtering individual items, use `where` prior to grouping.
|
||||
|
||||
> **warning** Inside a grouped query bare function names that match registered aggregates are treated as aggregates! To call a global function with the same name, use qualified access (e.g., `_G.sum(x)`).
|
||||
|
||||
# Aggregates without `group by`
|
||||
When an aggregate function appears in `select` or `having` but no `group by` clause is present, the entire result set is treated as a single implicit group. The `key` variable is `nil` in this case.
|
||||
|
||||
This is useful for computing a single summary value over a collection:
|
||||
|
||||
${query [[
|
||||
from p = index.pages()
|
||||
select { total = count(p.name), biggest = max(p.size) }
|
||||
]]}
|
||||
|
||||
A simple sum over a list:
|
||||
|
||||
${query [[
|
||||
from n = {10, 20, 30}
|
||||
select sum(n)
|
||||
]]}
|
||||
|
||||
The `having` clause also works without `group by` — it filters the single implicit group:
|
||||
|
||||
${query [[
|
||||
from n = {1, 2, 3}
|
||||
having sum(n) > 5
|
||||
select sum(n)
|
||||
]]}
|
||||
|
||||
> **note** Note
|
||||
> Without `group by`, the query always returns at most one row. If `having` rejects the implicit group, the result is empty.
|
||||
|
||||
# Available aggregates
|
||||
|
||||
All registered aggregate functions — built-in, user-defined, and aliases — can be listed via `index.aggregates()`:
|
||||
|
||||
${query[[
|
||||
select
|
||||
{
|
||||
Name = '`' .. name .. '`',
|
||||
Description = description,
|
||||
Kind =
|
||||
(builtin and 'builtin' or 'custom') ..
|
||||
(target and ' alias for ' .. '`' .. target .. '`' or ''),
|
||||
}
|
||||
from
|
||||
index.aggregates()
|
||||
order by
|
||||
builtin desc,
|
||||
name
|
||||
]]}
|
||||
|
||||
See [[Library/Std/APIs/Aggregate|Aggregate API]] for how to define custom aggregates and aliases.
|
||||
|
||||
# Examples
|
||||
All example queries operate on `tags.page`, but will work with any query collection. As always, to see the underlying query, hover over the result table and click the _Edit_ button to see the underlying query.
|
||||
|
||||
## Counting with and without binding
|
||||
Grouping pages by their first tag, and computing the count and aggregate statistics:
|
||||
|
||||
**Without binding variable**
|
||||
${query [[
|
||||
from
|
||||
tags.page
|
||||
group by
|
||||
tags[1]
|
||||
select {
|
||||
tag = key,
|
||||
total = count(name),
|
||||
min_size = min(size),
|
||||
max_size = max(size),
|
||||
avg_size = avg(size)
|
||||
}
|
||||
order by total desc
|
||||
]]}
|
||||
|
||||
**With binding variable**
|
||||
${query [[
|
||||
from
|
||||
p = tags.page
|
||||
group by
|
||||
p.tags[1]
|
||||
select {
|
||||
tag = key,
|
||||
total = count(p.name),
|
||||
min_size = min(p.size),
|
||||
max_size = max(p.size),
|
||||
avg_size = avg(p.size)
|
||||
}
|
||||
order by total desc
|
||||
]]}
|
||||
|
||||
## Multi-key grouping and aggregate
|
||||
${query[[
|
||||
from
|
||||
p = tags.page
|
||||
group by
|
||||
p.tags[1],
|
||||
p.tags[2]
|
||||
select {
|
||||
first = key[1],
|
||||
second = key[2],
|
||||
count = count(p.name)
|
||||
}
|
||||
]]}
|
||||
|
||||
## Group filtering with `having` and aggregates
|
||||
Only groups with more than two items and at least one tag set:
|
||||
${query[[
|
||||
from
|
||||
p = tags.page
|
||||
group by
|
||||
p.tags[1]
|
||||
having
|
||||
count(p.name) > 2 and key
|
||||
select {
|
||||
tag = key,
|
||||
total = count(p.name)
|
||||
}
|
||||
]]}
|
||||
|
||||
## Per-aggregate filtering with `filter(where ...)`
|
||||
Individual aggregate expressions can include a `filter(where <condition>)` clause to restrict which rows contribute to that specific aggregate.
|
||||
|
||||
Unlike `where` (which filters rows before grouping) and `having` (which filters entire groups after aggregation), `filter(where ...)` applies per-aggregate, per-row within each group. Multiple aggregates in the same `select` can each have different filters.
|
||||
|
||||
${query [[
|
||||
from
|
||||
p = index.pages()
|
||||
group by
|
||||
p.tags[1]
|
||||
select {
|
||||
tag = key,
|
||||
total = count(p.name),
|
||||
big = count(p.name) filter(where p.size > 10),
|
||||
big_sz = sum(p.size) filter(where p.size > 10)
|
||||
}
|
||||
order by
|
||||
tag
|
||||
]]}
|
||||
|
||||
The filter clause works with all aggregate functions: `count`, `sum`, `min`, `max`, `avg`, `array_agg`, and custom aggregates. When no rows match the filter condition, aggregates return their empty-group value: `0` for `count`, `nil` for `sum`, `min`, `max`, and `avg`, and an empty table `{}` for `array_agg`.
|
||||
|
||||
## Intra-aggregate `order by`
|
||||
Aggregate functions can include an `order by` clause **inside** the function call to control the order in which values are processed.
|
||||
|
||||
For commutative aggregates like `sum`, `count`, `min`, `max`, and `avg`, the intra-aggregate `order by` has no effect on the result because the value is the same regardless of iteration order. It is only meaningful for order-dependent aggregates like `array_agg`.
|
||||
|
||||
Ordered-set aggregates such as `quantile`, `percentile_cont`, and `percentile_disc` require an intra-aggregate `order by` clause to produce correct results, as they depend on the iteration order of input values. Without `order by`, results are undefined.
|
||||
|
||||
### Basic example
|
||||
|
||||
Collect page names sorted alphabetically within each group:
|
||||
|
||||
${query [[
|
||||
from
|
||||
p = index.pages()
|
||||
group by
|
||||
p.tags[1]
|
||||
select {
|
||||
tag = key,
|
||||
names_asc = array_agg(p.name order by p.name asc),
|
||||
names_desc = array_agg(p.name order by p.name desc)
|
||||
}
|
||||
order by
|
||||
tag
|
||||
limit
|
||||
5
|
||||
]]}
|
||||
|
||||
### Combined with `filter(where ...)`
|
||||
The `order by` and `filter` clauses can be used together. The filter is applied first (excluding rows), then the remaining rows are sorted before iteration:
|
||||
|
||||
${query [[
|
||||
from
|
||||
p = index.pages()
|
||||
group by
|
||||
p.tags[1]
|
||||
select {
|
||||
tag = key,
|
||||
big_by_size = array_agg(p.name order by p.size desc) filter(where p.size > 5)
|
||||
}
|
||||
order by
|
||||
tag
|
||||
limit
|
||||
5
|
||||
]]}
|
||||
|
||||
### Null handling
|
||||
The `nulls first` and `nulls last` modifiers work inside intra-aggregate `order by` the same way they do in the query-level `order by`:
|
||||
|
||||
```lua
|
||||
query [[
|
||||
from
|
||||
p = data
|
||||
group
|
||||
by p.category
|
||||
select {
|
||||
cat = key,
|
||||
items = array_agg(p.name
|
||||
order by
|
||||
p.priority asc nulls last
|
||||
)
|
||||
}
|
||||
]]
|
||||
```
|
||||
|
||||
## Field access after grouping
|
||||
Non-aggregated field references, such as `name` in `select`, refer to the first item in the group, matching common SQL and MySQL semantics.
|
||||
|
||||
${query [[
|
||||
from
|
||||
p = tags.page
|
||||
group by
|
||||
p.tags[1]
|
||||
select {
|
||||
tag = key,
|
||||
first_page = p.name,
|
||||
n = count(p.name)
|
||||
}
|
||||
]]}
|
||||
|
||||
## Custom aggregators
|
||||
Custom aggregator functions may be defined by the user using [[Library/Std/APIs/Aggregate|dedicated API]].
|
||||
|
||||
# See also
|
||||
* [[Space Lua/Integrated Query/Grouping]] — grouping queries without aggregation
|
||||
* [[Space Lua/Integrated Query]] — full SLIQ language reference and listing available aggregates
|
||||
@@ -0,0 +1,286 @@
|
||||
#maturity/experimental
|
||||
|
||||
The `group by` and `having` clauses extend [[Space Lua/Integrated Query]] with SQL-style grouping and aggregate filtering.
|
||||
|
||||
After `group by`, each result row has two fields:
|
||||
|
||||
- **`key`** - the group key (single value or table for multi-key)
|
||||
- **`group`** - a table (array) of all items in that group
|
||||
|
||||
The field names used in `group by` are also available as bare variables in `having`, `select`, and `order by`. Use `#group` to count items per group.
|
||||
|
||||
> **note** Note
|
||||
> `having` can only reference group key fields, `key`, `group`, aggregate expressions like `#group`, and aggregate functions like `count()`. To filter individual rows, use `where`.
|
||||
|
||||
# Examples
|
||||
|
||||
All examples below use `tags.tag`.
|
||||
|
||||
## Group by single key
|
||||
Group all tags by `name`:
|
||||
|
||||
${query [[
|
||||
from
|
||||
t = tags.tag
|
||||
group by
|
||||
t.name
|
||||
limit 5
|
||||
]]}
|
||||
|
||||
## Group by multiple keys
|
||||
Group tags by `name` and `parent` together:
|
||||
|
||||
${query[[
|
||||
from
|
||||
t = tags.tag
|
||||
group by
|
||||
t.name,
|
||||
t.parent
|
||||
limit 5
|
||||
]]}
|
||||
|
||||
## Filter groups by count
|
||||
Only show tags that appear more than 2 times:
|
||||
|
||||
${query[[
|
||||
from
|
||||
t = tags.tag
|
||||
group by
|
||||
t.name
|
||||
having
|
||||
#group > 2
|
||||
limit 5
|
||||
]]}
|
||||
|
||||
## Find unique tags
|
||||
Tags appearing exactly once:
|
||||
|
||||
${query[[
|
||||
from
|
||||
t = tags.tag
|
||||
group by
|
||||
t.name
|
||||
having
|
||||
#group == 1
|
||||
select key
|
||||
]]}
|
||||
|
||||
## Filter groups by key value
|
||||
Only show the group where `name` is "meta":
|
||||
|
||||
${query[[
|
||||
from
|
||||
tags.tag
|
||||
group by
|
||||
name
|
||||
having
|
||||
name == "meta"
|
||||
]]}
|
||||
|
||||
${query[[
|
||||
from
|
||||
t = tags.tag
|
||||
group by
|
||||
t.name
|
||||
having
|
||||
t.name == "meta"
|
||||
]]}
|
||||
|
||||
## Multi-key having
|
||||
|
||||
Groups by `name` and `parent`, keep only page-level tags with more than 1 entry:
|
||||
|
||||
${query [[
|
||||
from
|
||||
tags.tag
|
||||
group by
|
||||
name,
|
||||
parent
|
||||
having
|
||||
parent == 'page' and
|
||||
#group > 1
|
||||
]]}
|
||||
|
||||
## `where` before `group by`
|
||||
|
||||
Filter to page parents first, then group by `name`:
|
||||
|
||||
${query [[
|
||||
from
|
||||
index.tags()
|
||||
where
|
||||
parent == 'page'
|
||||
group by
|
||||
name
|
||||
]]}
|
||||
|
||||
## `where`, `group by` and `having` combined
|
||||
|
||||
Filter to page parents, group by `name`, keep groups with 2+ items:
|
||||
${query [[
|
||||
from
|
||||
index.tags()
|
||||
where
|
||||
parent == 'page'
|
||||
group by
|
||||
name
|
||||
having
|
||||
#group >= 2
|
||||
]]}
|
||||
|
||||
## `select` name and count
|
||||
|
||||
Project each group into a table with `name` and `count`:
|
||||
${query [[
|
||||
from
|
||||
index.tags()
|
||||
group by
|
||||
name
|
||||
select {
|
||||
name = name,
|
||||
count = #group
|
||||
}
|
||||
]]}
|
||||
|
||||
## `select` with multi-key
|
||||
|
||||
Project both key parts and count:
|
||||
|
||||
${query [[
|
||||
from
|
||||
index.tags()
|
||||
group by
|
||||
name,
|
||||
parent
|
||||
select {
|
||||
name = name,
|
||||
parent = parent,
|
||||
count = #group
|
||||
}
|
||||
]]}
|
||||
|
||||
## Full pipeline: `where`, `group by`, `having` and `select`
|
||||
|
||||
Filter, group, filter groups, then project:
|
||||
|
||||
${query [[
|
||||
from
|
||||
index.tags()
|
||||
where
|
||||
parent == 'page' or
|
||||
parent == 'task'
|
||||
group by
|
||||
name
|
||||
having
|
||||
#group > 1
|
||||
select {
|
||||
tag = name,
|
||||
total = #group
|
||||
}
|
||||
]]}
|
||||
|
||||
## Order groups by count
|
||||
|
||||
Sort groups by size, largest first:
|
||||
|
||||
${query [[
|
||||
from
|
||||
index.tags()
|
||||
group by
|
||||
name
|
||||
order by
|
||||
#group desc
|
||||
]]}
|
||||
|
||||
## Top tags with `having`, `order by`, and `select`
|
||||
|
||||
Tags with 2+ occurrences, sorted by count, projected:
|
||||
${query [[
|
||||
from
|
||||
index.tags()
|
||||
group by
|
||||
name
|
||||
having
|
||||
#group >= 2
|
||||
order by
|
||||
#group desc
|
||||
select {
|
||||
tag = name,
|
||||
count = #group
|
||||
}
|
||||
]]}
|
||||
|
||||
## Top N groups with `limit`
|
||||
${query [[
|
||||
from
|
||||
index.tags()
|
||||
group by
|
||||
name
|
||||
order by
|
||||
#group desc
|
||||
limit
|
||||
3
|
||||
]]}
|
||||
|
||||
## Full pipeline with `limit`
|
||||
Top 5 tags with 2+ uses, showing name and count:
|
||||
${query [[
|
||||
from
|
||||
index.tags()
|
||||
group by
|
||||
p.name
|
||||
having
|
||||
#group > 1
|
||||
select {
|
||||
tag = name,
|
||||
count = #group
|
||||
}
|
||||
]]}
|
||||
|
||||
## Multi-key with explicit object variable
|
||||
Full pipeline with `p =` binding and two group keys:
|
||||
|
||||
${query [[
|
||||
from
|
||||
index.tags()
|
||||
where
|
||||
p.parent == 'page'
|
||||
group by
|
||||
p.name,
|
||||
p.parent
|
||||
having
|
||||
#group >= 2
|
||||
order by
|
||||
#group desc
|
||||
select {
|
||||
tag = name,
|
||||
parent = parent,
|
||||
count = #group
|
||||
}
|
||||
]]}
|
||||
|
||||
## Access `key` directly
|
||||
|
||||
For single-key grouping, `key` holds the value directly:
|
||||
${query [[
|
||||
from
|
||||
index.tags()
|
||||
group by
|
||||
name
|
||||
having
|
||||
key == 'meta'
|
||||
]]}
|
||||
|
||||
## Access `key` table for multi-key
|
||||
|
||||
For multi-key grouping, `key` is a table indexed from 1:
|
||||
${query [[
|
||||
from
|
||||
index.tags()
|
||||
group by
|
||||
name,
|
||||
parent
|
||||
having
|
||||
key[1] == 'meta' and
|
||||
key[2] == 'page'
|
||||
]]}
|
||||
Reference in New Issue
Block a user