Personal outliner built with Rust.
add trawler-mvp change: proposal, design, specs, tasks
graham.systems
(Jul 12, 2026, 2:52 PM -0700)
ffb1d89a
ea200b8b
+514
+3
openspec/changes/trawler-mvp/README.md
+3
openspec/changes/trawler-mvp/README.md
+117
openspec/changes/trawler-mvp/design.md
+117
openspec/changes/trawler-mvp/design.md
···
1
+
# Design: trawler-mvp
2
+
3
+
## Context
4
+
5
+
Greenfield repository. Trawler is a single-user, keyboard-first outliner in the mold of Logseq DB, built Rust-native. The developer's primary machine is Windows 11; the UI targets GPUI (Zed's framework), whose Windows backend is its youngest. The project deliberately trades ecosystem maturity for feel: GPU-rendered UI, embedded Scheme queries, CRDT-native storage.
6
+
7
+
Design conversation history: the architecture below emerged from an explore session that evaluated the embedded graph-DB landscape (Kuzu's abandonment after the Apple acquisition of Kùzu Inc. was a formative cautionary tale), Logseq's file-first → DB-first migration pain, and the Zed/Helix precedents for GPUI and Steel respectively.
8
+
9
+
## Goals / Non-Goals
10
+
11
+
**Goals:**
12
+
13
+
- Daily-driver quality outliner: instant startup, 60fps+ interaction, keyboard-complete.
14
+
- Logseq DB semantics: everything (blocks, pages, tags, dates) is a node in one graph.
15
+
- Journal-centric: the daily note is the front door.
16
+
- Queries as a first-class product surface: written in Steel (Scheme), edited with real language tooling, previewed live.
17
+
- Sync-*ready* storage from day one, without shipping sync.
18
+
- All indexes disposable: any derived structure can be deleted and rebuilt from the source of truth.
19
+
20
+
**Non-Goals:**
21
+
22
+
- Multiplayer/collaboration. CRDT storage is for sync and undo, not concurrent editing UX.
23
+
- Sync transport (a future change; the storage format is the enabler).
24
+
- Semantic/embedding similarity (lexical only in MVP; architecture leaves a seam).
25
+
- Steel as a general plugin system (queries only; write-capable scripting is a future, deliberate decision).
26
+
- File-first Markdown storage. Plain-Markdown *export* is future work; `.md` files are never the source of truth.
27
+
- Mobile, web, CLI capture.
28
+
29
+
## Decisions
30
+
31
+
### D1: Loro document as the single source of truth
32
+
33
+
The durable record is a Loro CRDT document: the outline as Loro's movable tree, block content as Loro text, block metadata (properties, tags) as Loro maps. Persistence is snapshot + incremental update blobs on the filesystem (plain files in the graph directory; no database owns the truth).
34
+
35
+
- **Why**: Loro's movable tree is purpose-built for outliner reparenting. CRDT storage makes future sync = "exchange update blobs over any dumb transport" (even a Syncthing folder) with zero merge code. History/undo comes with the format. Loro is Rust-native and judged stable enough.
36
+
- **Alternatives considered**:
37
+
- *SQLite as truth* — boring-reliable, but sync becomes a real project later and block identity/history must be hand-built.
38
+
- *Markdown files as truth* — rejected; Logseq spent years escaping this (startup reparse, block-ID pollution, merge hell).
39
+
40
+
### D2: All query structures are derived, disposable indexes
41
+
42
+
An in-memory graph (adjacency, backlink sets, tag/property indexes, date index) is rebuilt from the Loro doc at startup and updated incrementally on edit. Tantivy's index lives on disk but is equally disposable. Deleting the index directory must never lose data.
43
+
44
+
- **Why**: converts the risky "which graph DB do I marry?" decision into a swappable one. The post-Kuzu landscape (archived overnight; forks immature) proved young embedded graph DBs are dependencies you might outlive.
45
+
- **Alternatives considered**:
46
+
- *Grafeo* — impressive breadth (LPG+RDF, many query languages, HNSW), but very young/single-vendor, and its headline query languages are worthless here since Steel is the user surface. Candidate for a later audition (its HNSW could serve semantic similarity).
47
+
- *OverGraph* — API-first, philosophically closer (we need a Rust API, not a query language), but young and aimed at AI-agent memory. Also a later candidate if index rebuild time ever hurts.
48
+
- *CozoDB* — Datalog lineage is attractive; maintenance has gone quiet.
49
+
- *SQLite* — remains the boring fallback if the in-memory index ever needs persistence.
50
+
- **Owning the in-memory graph wins for a Steel-specific reason**: query quality depends on how tightly Scheme primitives map to index operations. Owning the structure lets `(and ...)` compile to native set intersection rather than composing through a third party's API assumptions.
51
+
52
+
### D3: GPUI + gpui-component for the shell; one-hot block editor
53
+
54
+
Exactly one live text input exists at any time — the focused block. All other blocks are read-only rendered views (styled Markdown). Focus movement detaches the editor, re-renders the block, reattaches to the target.
55
+
56
+
- **Why one-hot**: this is Logseq's own pattern and the cheat code that makes a native-GUI outliner tractable — one good single-block input (gpui-component provides the base) plus a fast rendered view, instead of a Zed-grade multi-buffer editor.
57
+
- **Why GPUI**: "feels like Zed" is a stated requirement, and GPUI *is* that feel. `gpui-component` supplies inputs, lists, scrollbars. Helix/Zed precedents exist for the tree-sitter integration Steel editing needs.
58
+
- **Risk accepted**: GPUI's Windows backend is its youngest and docs are thin; the developer is on Windows. Gated by Spike S1.
59
+
60
+
### D4: Steel (Scheme) as the query language, capability-scoped
61
+
62
+
Query blocks are outline blocks whose content is a Steel expression. They evaluate in a VM context where only a read-only graph API is registered as native functions. Results render inline (as an outline slice or table) and re-evaluate on a debounce as the graph or the query changes.
63
+
64
+
- **Why Steel**: user requirement — a real Lisp, not a bolted-on DSL. Steel is designed for Rust embedding, has an LSP (`steel-language-server`) and a Helix-precedented editor integration; the Logseq lineage (ClojureScript/DataScript) survives in spirit.
65
+
- **Capability scoping**: queries get read-only natives only. If Steel later becomes an extension language, write access lives in a separate deliberate context. Decided now to prevent scope creep.
66
+
- **"Scheme composes, Rust iterates"**: the query API exposes set-algebra primitives (blocks-with-tag, backlinks-of, in-date-range → native set handles; and/or/not → native intersection/union/difference). Steel lambdas never run per-block over the whole graph in the hot path. Per-block predicates are allowed only over already-narrowed sets. This is an API design decision, not an optimization pass.
67
+
- **Runaway queries**: eval runs off the UI thread with cancellation/time-boxing. Whether Steel supports clean interruption is unproven — gated by Spike S2.
68
+
69
+
### D5: Journal timeline is the home view
70
+
71
+
Opening the app lands on today's journal page (auto-created), with previous days scrolling behind it. Date references (`[[2026-07-02]]`-style) are first-class node references to journal pages.
72
+
73
+
### D6: Tantivy for search; lexical similarity for "similar blocks"
74
+
75
+
Full-text search over block content via tantivy. "Similar blocks" uses lexical similarity (shared rare terms — more-like-this style) — zero ML dependency, surprisingly good on notes. A semantic tier (local embeddings, brute-force cosine at personal scale) is future work; the similarity provider is a trait so the seam exists.
76
+
77
+
### D7: Full Markdown block content
78
+
79
+
Blocks contain full Markdown: multiple paragraphs, emphasis, code spans and fenced code blocks, quotes, embedded lists, and links — matching Logseq DB, where a node is a fully formatted document fragment. Node references (`[[page]]`, `#tag`, dates, block refs) are an inline extension parsed alongside Markdown.
80
+
81
+
- **Why**: user requirement and Logseq DB parity; constraining content to an inline subset was considered and rejected — it saves renderer work but breaks the mental model users bring from Logseq/Obsidian.
82
+
- **Consequences**:
83
+
- The one-hot editor is a small multi-line editor, not a single-line input. **Enter creates a new sibling block; Shift+Enter inserts a newline/paragraph within the block** (Logseq's convention).
84
+
- Rendering uses a real Markdown parser per block (pulldown-cmark or tree-sitter-markdown) with an inline extension for node references.
85
+
- Node references SHALL NOT be parsed inside code spans or fenced code blocks.
86
+
- Markdown structure inside a block (list items, paragraphs) is *content*, not outline: only blocks are nodes — referenceable, foldable, movable. The outline tree remains the sole hierarchy.
87
+
88
+
### D8: Workspace layout
89
+
90
+
```
91
+
crates/
92
+
trawler-core/ # block graph, Loro storage, indexes, Steel query engine, search
93
+
# no UI dependencies; fully testable headless
94
+
trawler/ # GPUI app: outline editor, journal view, navigation, query UI
95
+
```
96
+
97
+
Core/UI split keeps the engine testable without GPUI and leaves room for future frontends (CLI capture) without commitment.
98
+
99
+
## Risks / Trade-offs
100
+
101
+
- [GPUI Windows backend is young; breakage or missing pieces on the primary dev machine] → Spike S1 first: stand up a GPUI window with gpui-component input on Windows before any other UI work. Fallback documented if blocked: re-evaluate frontend (Tauri) rather than abandon — core is UI-independent.
102
+
- [Steel eval can't be cleanly interrupted → runaway query freezes previews] → Spike S2: prove background-thread eval with cancellation/timeout. Fallback: run query VM in a dedicated thread that can be dropped/respawned; queries are stateless so a killed VM costs nothing.
103
+
- [Loro doc grows unboundedly / rebuild-at-startup becomes slow at real scale] → Spike S3: synthetic 100k-block graph; measure snapshot size, load time, index rebuild time. Loro supports snapshot compaction; budget: cold start under ~1s at 100k blocks.
104
+
- [One-hot editor feels laggy on focus transitions (detach/render/reattach per keystroke navigation)] → treat editor transition latency as a benchmark from the first prototype; pre-render adjacent blocks if needed.
105
+
- [gpui/gpui-component API churn (pre-1.0)] → pin revisions; vendor patches if needed; upgrade deliberately, not continuously.
106
+
- [Steel pre-1.0 API changes] → pin; the query API surface is ours, so churn is contained to the embedding layer.
107
+
- [Everything-is-derived means index bugs silently corrupt query results] → index rebuild is cheap and deterministic: golden tests compare incremental index state against full rebuild after randomized edit sequences.
108
+
109
+
## Migration Plan
110
+
111
+
Greenfield — no migration. Rollback story for users is inherent: the Loro doc directory is the data; deleting derived indexes is always safe.
112
+
113
+
## Open Questions
114
+
115
+
- **Query surface shape**: datalog-style relations as Scheme macros vs. functional set combinators (current lean: combinators first — they fall out of the API design; a relational macro layer can be pure Scheme on top later).
116
+
- **Loro granularity**: one doc for the whole graph vs. doc-per-page. Current lean: one doc (simpler identity, atomic cross-page moves); Spike S3 validates at scale.
117
+
- **Similar-blocks UX**: where do suggestions surface (dedicated panel vs. inline footer on focused block)? Defer to prototype feedback.
+39
openspec/changes/trawler-mvp/proposal.md
+39
openspec/changes/trawler-mvp/proposal.md
···
1
+
# Proposal: trawler-mvp
2
+
3
+
## Why
4
+
5
+
Existing outliners force a trade-off: Logseq DB has the right data model (blocks as graph nodes, journal-centric, query-driven) but is built on Electron and carries years of file-era compromises; native editors like Zed feel instant but don't do networked thought. Trawler is a personal daily-driver outliner that takes Logseq DB's semantics and rebuilds them Rust-native: a GPUI shell that feels like Zed, a CRDT-backed block graph that makes sync a property of the storage format rather than a feature, and a real Lisp (Steel/Scheme) as the query language instead of a bolted-on DSL.
6
+
7
+
## What Changes
8
+
9
+
- New cargo workspace with a core/UI split: `trawler-core` (block graph, storage, queries, search — no UI dependencies) and `trawler` (GPUI application shell).
10
+
- Block-based data model where everything is a node: blocks, pages, tags, and journal dates are all entities in one graph, with references and lightweight typed properties.
11
+
- Loro (CRDT) document as the durable source of truth; all query/search structures are derived, disposable indexes rebuilt from it.
12
+
- GPUI outline editor using the one-hot editor pattern: exactly one live text input (the focused block), everything else rendered read-only.
13
+
- Journal-first workflow: auto-created daily pages, date references, chronological timeline as the default landing view.
14
+
- Keyboard-driven graph traversal: follow references, jump to backlinks, zoom into/out of blocks, navigate without the mouse.
15
+
- Steel (Scheme) embedded as the query language: query blocks live in the outline, evaluated against a read-only graph API, with syntax-highlighted Lisp editing and live result previews.
16
+
- Full-text search over all blocks (tantivy) plus lexical "similar blocks" suggestions.
17
+
- Sync-ready persistence: snapshot + update-log storage format so multi-device sync can later ship as update-blob exchange with no schema change.
18
+
19
+
## Capabilities
20
+
21
+
### New Capabilities
22
+
23
+
- `block-graph`: The core data model — blocks, outline hierarchy, pages, references, tags, properties; Loro-backed persistence and the derived in-memory graph with backlink indexes.
24
+
- `outline-editor`: The GPUI editing surface — one-hot block editing, keyboard-first outline manipulation (indent, outdent, move, fold), markdown rendering of inactive blocks.
25
+
- `journal`: Daily notes — automatic date pages, date references, and the journal timeline as the app's home view.
26
+
- `graph-navigation`: Moving through the graph — following references, backlinks panels, block zoom, keyboard hopping between linked nodes.
27
+
- `steel-queries`: The Steel VM integration — read-only query API, query blocks with live previews, and the in-app Lisp editing experience.
28
+
- `search`: Full-text search and lexical similar-content suggestions over the block graph.
29
+
30
+
### Modified Capabilities
31
+
32
+
_None — greenfield project; no existing specs._
33
+
34
+
## Impact
35
+
36
+
- **New code**: entire repository is new. Cargo workspace: `crates/trawler-core`, `crates/trawler` (GPUI app).
37
+
- **Key dependencies**: `gpui` + `gpui-component` (UI; youngest platform is Windows — the primary dev machine), `loro` (CRDT storage), `steel-core` (embedded Scheme), `tantivy` (search), `tree-sitter` + Scheme grammar (query highlighting), `pulldown-cmark` or tree-sitter-markdown (block rendering).
38
+
- **Platform risk**: GPUI on Windows and Steel eval interruption are unproven for this use; both are gated behind explicit early spikes in tasks.
39
+
- **Out of scope for this change**: multi-device sync transport (the storage format enables it; shipping it is a future change), semantic/embedding similarity (lexical only for now), Steel as a general extension/plugin language (queries get a read-only API; write-capable scripting is a deliberate future decision), CLI quick-capture, mobile.
+58
openspec/changes/trawler-mvp/specs/block-graph/spec.md
+58
openspec/changes/trawler-mvp/specs/block-graph/spec.md
···
1
+
# block-graph
2
+
3
+
The core data model: blocks, outline hierarchy, pages, references, tags, and properties, with Loro-backed persistence and derived in-memory indexes.
4
+
5
+
## ADDED Requirements
6
+
7
+
### Requirement: Everything is a node
8
+
The system SHALL represent blocks, pages, tags, and journal dates as nodes in a single graph. Pages SHALL be nodes that own an ordered tree of block nodes; tags and dates SHALL be referenceable nodes with no special-cased storage.
9
+
10
+
#### Scenario: A tag is navigable like a page
11
+
- **WHEN** a block is tagged `#project`
12
+
- **THEN** `project` exists as a node whose backlinks include that block, exactly as a `[[project]]` page reference would
13
+
14
+
### Requirement: Stable block identity
15
+
Every block SHALL have a globally unique, stable identifier that survives edits, moves, reparenting, application restarts, and (future) sync merges.
16
+
17
+
#### Scenario: Identity survives restructuring
18
+
- **WHEN** a block is edited, indented under a new parent, and the application is restarted
19
+
- **THEN** the block retains its original identifier and all references to it resolve
20
+
21
+
### Requirement: Outline tree operations
22
+
The system SHALL support creating, deleting, splitting, and merging blocks, and moving blocks (with their subtrees) to any position under any parent, preserving sibling order.
23
+
24
+
#### Scenario: Subtree move
25
+
- **WHEN** a block with three descendant blocks is moved under a different parent
26
+
- **THEN** the block and all three descendants appear at the new location in their original relative order, and the source location no longer contains them
27
+
28
+
### Requirement: Loro document is the source of truth
29
+
The system SHALL persist the graph exclusively as a Loro CRDT document (movable tree for hierarchy, text for block content, maps for metadata), stored as a snapshot plus incremental update blobs in the graph directory. No derived structure may hold data that cannot be rebuilt from the Loro document.
30
+
31
+
#### Scenario: Derived indexes are disposable
32
+
- **WHEN** all derived index files are deleted and the application is restarted
33
+
- **THEN** the graph loads completely from the Loro document and all queries, backlinks, and search results are identical to their pre-deletion state
34
+
35
+
#### Scenario: Crash safety
36
+
- **WHEN** the process is killed immediately after an edit is acknowledged in the UI
37
+
- **THEN** on next startup the edit is present
38
+
39
+
### Requirement: Block references and backlinks
40
+
Block content SHALL support inline references to pages, tags, dates, and individual blocks. The system SHALL maintain a backlink index mapping every node to the set of blocks that reference it, updated within the same edit transaction.
41
+
42
+
#### Scenario: Backlink appears immediately
43
+
- **WHEN** the user types a reference to `[[fishing]]` inside a block and confirms it
44
+
- **THEN** the `fishing` node's backlink set includes the referencing block without an application restart or manual reindex
45
+
46
+
### Requirement: Lightweight properties
47
+
Blocks SHALL support key–value properties where values are typed as text, number, date, boolean, or node reference. Property keys SHALL be queryable across the graph. Property schemas, classes, and inheritance are explicitly out of scope for this change.
48
+
49
+
#### Scenario: Property round-trip
50
+
- **WHEN** a block is given the property `due` with the date value `2026-07-10`
51
+
- **THEN** a query for blocks with a `due` property returns the block, and the value is retrieved as a date (not a string)
52
+
53
+
### Requirement: Incremental index maintenance
54
+
The in-memory graph and all derived indexes SHALL be updated incrementally on every edit, and an incremental state SHALL be behaviorally indistinguishable from a full rebuild.
55
+
56
+
#### Scenario: Incremental equals rebuild
57
+
- **WHEN** a randomized sequence of at least 1,000 edit operations (create, edit, move, delete, tag, reference) is applied
58
+
- **THEN** the incrementally-maintained indexes are equal to indexes rebuilt from scratch from the Loro document
+30
openspec/changes/trawler-mvp/specs/journal/spec.md
+30
openspec/changes/trawler-mvp/specs/journal/spec.md
···
1
+
# journal
2
+
3
+
Daily notes: automatic date pages, date references, and the journal timeline as the application's home view.
4
+
5
+
## ADDED Requirements
6
+
7
+
### Requirement: Automatic daily pages
8
+
The system SHALL ensure a journal page exists for the current local date whenever the application opens or the date rolls over while running. Journal pages SHALL be ordinary page nodes distinguished by a date attribute, created lazily and never duplicated.
9
+
10
+
#### Scenario: First launch of the day
11
+
- **WHEN** the application opens on a date with no existing journal page
12
+
- **THEN** a journal page for that date exists and is focused with an empty first block ready for input
13
+
14
+
#### Scenario: No empty-page litter
15
+
- **WHEN** a journal page was auto-created but never received content
16
+
- **THEN** it is not shown in the timeline on subsequent days
17
+
18
+
### Requirement: Journal timeline is the home view
19
+
The application SHALL open to a chronological timeline with today's journal page at the top and previous non-empty journal pages loading below it as the user scrolls.
20
+
21
+
#### Scenario: Opening the app
22
+
- **WHEN** the application starts
23
+
- **THEN** today's journal page is displayed with focus in its first block, and scrolling down reveals prior days in reverse-chronological order
24
+
25
+
### Requirement: Date references resolve to journal pages
26
+
Date references in block content SHALL be node references to the corresponding journal page, participating in backlinks like any other reference. Typing support SHALL make date references cheap to produce (e.g., a date picker or natural-language completion).
27
+
28
+
#### Scenario: Scheduling by reference
29
+
- **WHEN** a block on any page references `[[2026-07-10]]`
30
+
- **THEN** the journal page for 2026-07-10 lists that block in its backlinks, visible when that day is viewed
+56
openspec/changes/trawler-mvp/specs/outline-editor/spec.md
+56
openspec/changes/trawler-mvp/specs/outline-editor/spec.md
···
1
+
# outline-editor
2
+
3
+
The GPUI editing surface: one-hot block editing, keyboard-first outline manipulation, and rendered display of inactive blocks.
4
+
5
+
## ADDED Requirements
6
+
7
+
### Requirement: One-hot block editing
8
+
The application SHALL maintain at most one live text input at any time — the focused block. All other blocks SHALL be rendered as read-only styled views. Moving focus SHALL commit the current block's content, re-render it, and attach the editor to the target block.
9
+
10
+
#### Scenario: Focus transition commits content
11
+
- **WHEN** the user edits a block and presses Down to focus the next block
12
+
- **THEN** the edited block displays its new rendered content and the next block becomes the sole editable input, with no intermediate state where zero or two editors exist
13
+
14
+
### Requirement: Keyboard-complete outline manipulation
15
+
All outline operations SHALL be executable without the mouse: create sibling (Enter), insert a newline within the block (Shift+Enter), split block at cursor, indent/outdent (Tab/Shift+Tab), move block up/down among siblings, delete/merge with previous (Backspace at start), and fold/unfold subtree.
16
+
17
+
#### Scenario: Indent under previous sibling
18
+
- **WHEN** the cursor is in a block and the user presses Tab
19
+
- **THEN** the block (with its subtree) becomes the last child of its previous sibling, and the cursor position within the text is preserved
20
+
21
+
#### Scenario: Newline within a block vs. new block
22
+
- **WHEN** the user presses Shift+Enter mid-block, types a second paragraph, then presses Enter
23
+
- **THEN** the block contains both paragraphs, and a new empty sibling block is created and focused
24
+
25
+
#### Scenario: Fold hides descendants
26
+
- **WHEN** the user folds a block with descendants
27
+
- **THEN** descendants are hidden, a fold indicator is shown, and keyboard navigation skips the hidden blocks
28
+
29
+
### Requirement: Markdown rendering of block content
30
+
Inactive blocks SHALL render their content as full Markdown — multiple paragraphs, emphasis, code spans, fenced code blocks, quotes, embedded lists, and links — plus node references as an inline extension. Node references SHALL be visually distinct, SHALL identify their target, and SHALL NOT be parsed inside code spans or fenced code blocks. Markdown structure within a block is content only: it SHALL NOT create outline nodes.
31
+
32
+
#### Scenario: Reference renders as a link
33
+
- **WHEN** a block containing `[[gear-maintenance]]` is not focused
34
+
- **THEN** the reference renders as a styled link displaying the target page's name
35
+
36
+
#### Scenario: Multi-paragraph block renders fully
37
+
- **WHEN** a block containing two paragraphs and a fenced code block is not focused
38
+
- **THEN** both paragraphs and the syntax-styled code fence render within the single block, which folds, moves, and is referenced as one node
39
+
40
+
#### Scenario: References in code are literal
41
+
- **WHEN** a block's fenced code block contains the text `[[not-a-link]]`
42
+
- **THEN** it renders as literal code, creates no backlink, and `not-a-link` gains no referencing node
43
+
44
+
### Requirement: Editing latency
45
+
Focus transitions between blocks SHALL complete within one frame at 60Hz (under ~16ms) on the development machine, measured from keypress to the new editor accepting input, for outlines of at least 10,000 visible-tree blocks.
46
+
47
+
#### Scenario: Rapid navigation stays responsive
48
+
- **WHEN** the user holds Down to traverse 50 consecutive blocks
49
+
- **THEN** focus tracks the keypress rate with no perceptible lag or dropped keystrokes
50
+
51
+
### Requirement: Large outlines render lazily
52
+
The outline view SHALL virtualize rendering such that memory and frame time scale with visible blocks, not total blocks in the page.
53
+
54
+
#### Scenario: Long journal page scrolls smoothly
55
+
- **WHEN** a page contains 10,000 blocks and the user scrolls through it
56
+
- **THEN** scrolling maintains 60fps and only visible blocks are materialized as UI elements
+40
openspec/changes/trawler-mvp/specs/search/spec.md
+40
openspec/changes/trawler-mvp/specs/search/spec.md
···
1
+
# search
2
+
3
+
Full-text search and lexical similar-content suggestions over the block graph.
4
+
5
+
## ADDED Requirements
6
+
7
+
### Requirement: Full-text search over all blocks
8
+
The system SHALL index the text content of every block (tantivy) and provide keyboard-invoked search returning matching blocks ranked by relevance, with match highlighting and navigation to the source block.
9
+
10
+
#### Scenario: Search and jump
11
+
- **WHEN** the user invokes search and types `otter trawl`
12
+
- **THEN** blocks containing the terms are listed ranked by relevance with matches highlighted, and confirming an entry navigates to that block in its page
13
+
14
+
### Requirement: Index freshness
15
+
Newly created or edited block content SHALL be searchable within one second of the edit, without manual reindexing.
16
+
17
+
#### Scenario: Just-typed content is findable
18
+
- **WHEN** the user writes a block containing a novel term and invokes search for it within a second
19
+
- **THEN** the block appears in the results
20
+
21
+
### Requirement: Search is available to queries
22
+
Full-text search SHALL be exposed as a set-returning primitive in the Steel query API, composable with other set primitives.
23
+
24
+
#### Scenario: Search composed with a tag filter
25
+
- **WHEN** a query intersects full-text results for `"bycatch"` with the `#research` tag set
26
+
- **THEN** only blocks matching both are returned
27
+
28
+
### Requirement: Similar blocks (lexical)
29
+
The system SHALL suggest blocks lexically similar to a given block (shared distinctive terms, more-like-this style), excluding the block itself and its ancestors/descendants, ranked by similarity. The similarity provider SHALL be a swappable interface so a semantic (embedding) tier can be added later without changing consumers.
30
+
31
+
#### Scenario: Related note resurfaces
32
+
- **WHEN** the user views similar blocks for a block about "mending the cod-end after the last haul"
33
+
- **THEN** previously written blocks sharing distinctive vocabulary (e.g., other cod-end or net-repair notes) are listed and navigable, and unrelated blocks are not
34
+
35
+
### Requirement: Search index is disposable
36
+
The search index SHALL be entirely derivable from the block graph. Deleting the index directory SHALL trigger a transparent rebuild on next startup with no data loss.
37
+
38
+
#### Scenario: Rebuild after deletion
39
+
- **WHEN** the search index directory is deleted and the application restarts
40
+
- **THEN** search returns the same results as before deletion once the rebuild completes
+58
openspec/changes/trawler-mvp/specs/steel-queries/spec.md
+58
openspec/changes/trawler-mvp/specs/steel-queries/spec.md
···
1
+
# steel-queries
2
+
3
+
The Steel (Scheme) query integration: read-only query API, query blocks with live previews, and the in-app Lisp editing experience.
4
+
5
+
## ADDED Requirements
6
+
7
+
### Requirement: Query blocks live in the outline
8
+
The system SHALL support a query block: an outline block whose content is a Steel expression and which renders its evaluation result inline beneath the expression. Query blocks SHALL be ordinary blocks (movable, referenceable, foldable).
9
+
10
+
#### Scenario: Results render in place
11
+
- **WHEN** a query block contains `(blocks (and (tag 'project) (ref "rust")))`
12
+
- **THEN** the matching blocks render beneath the query as a navigable outline slice, each entry linking to its source block
13
+
14
+
### Requirement: Read-only capability scoping
15
+
Query evaluation SHALL run in a Steel context where only read-only graph primitives are registered. No mutation of the graph, filesystem access, or network access SHALL be reachable from a query.
16
+
17
+
#### Scenario: Mutation is unavailable
18
+
- **WHEN** a query attempts to call any mutating or I/O operation
19
+
- **THEN** evaluation fails with an unknown-identifier error and the graph is unchanged
20
+
21
+
### Requirement: Set-algebra query primitives
22
+
The query API SHALL expose native set-returning primitives — at minimum: blocks by tag, blocks referencing a node, blocks with a property (optionally matching a value), blocks within a date range, descendants of a node, and full-text search results — plus native `and`/`or`/`not` combinators that operate on sets without per-block Steel callbacks.
23
+
24
+
#### Scenario: Native combination stays fast
25
+
- **WHEN** a query intersects a 20,000-block tag set with a 5,000-block date-range set on a 100,000-block graph
26
+
- **THEN** evaluation completes within 50ms, with the intersection performed natively rather than by iterating blocks in Scheme
27
+
28
+
#### Scenario: Scheme predicates over narrowed sets
29
+
- **WHEN** a query applies a Steel lambda filter to a set already narrowed to under 1,000 blocks
30
+
- **THEN** the per-block predicate is applied and results are correct
31
+
32
+
### Requirement: Live preview with debounce
33
+
While a query block is being edited, the system SHALL re-evaluate it on a debounce and update the rendered results without requiring the user to leave the block.
34
+
35
+
#### Scenario: Editing refines results live
36
+
- **WHEN** the user extends `(tag 'project)` to `(and (tag 'project) (since "2026-06"))` and pauses typing
37
+
- **THEN** the preview updates to the narrowed result set without explicit re-run
38
+
39
+
### Requirement: Runaway query containment
40
+
Query evaluation SHALL run off the UI thread and SHALL be cancellable. A query exceeding its time budget SHALL be terminated and reported as timed out; the UI SHALL remain responsive throughout.
41
+
42
+
#### Scenario: Infinite loop cannot freeze the app
43
+
- **WHEN** a query block contains a non-terminating expression
44
+
- **THEN** the UI continues to respond, the evaluation is terminated at the time budget, and the query block displays a timeout error
45
+
46
+
### Requirement: Lisp editing experience
47
+
When editing a query block, the editor SHALL provide Scheme syntax highlighting (tree-sitter), matching-paren indication, and inline display of evaluation errors with position information where available.
48
+
49
+
#### Scenario: Error is shown at the block
50
+
- **WHEN** a query contains an unbalanced paren or unknown identifier
51
+
- **THEN** the query block displays the error message inline instead of results, and the rest of the outline is unaffected
52
+
53
+
### Requirement: Table rendering for property queries
54
+
Query results SHALL render as an outline slice by default, and as a table when the query projects named properties.
55
+
56
+
#### Scenario: Projected properties become columns
57
+
- **WHEN** a query projects each matching block's content and its `due` property
58
+
- **THEN** results render as a two-column table with one row per block, sortable by column
+71
openspec/changes/trawler-mvp/tasks.md
+71
openspec/changes/trawler-mvp/tasks.md
···
1
+
# Tasks: trawler-mvp
2
+
3
+
## 1. Workspace & Spikes (de-risk the bets before building on them)
4
+
5
+
- [ ] 1.1 Create cargo workspace: `crates/trawler-core`, `crates/trawler`; pin toolchain; CI check/test/clippy on Windows
6
+
- [ ] 1.2 Spike S1 — GPUI on Windows: window + gpui-component text input + virtualized list renders and accepts input at 60fps on the dev machine; record pinned revisions and any workarounds
7
+
- [ ] 1.3 Spike S2 — Steel interruption: evaluate Steel expressions on a background thread with cancellation and a time budget; prove a `(loop)` query cannot block or leak; document the containment pattern (cooperative interrupt vs droppable VM thread)
8
+
- [ ] 1.4 Spike S3 — Loro at scale: generate a synthetic 100k-block graph; measure snapshot size, load time, and full index rebuild; confirm cold start < ~1s and settle the one-doc-vs-doc-per-page question in design.md
9
+
- [ ] 1.5 Go/no-go checkpoint: fold spike findings back into design.md (update Decisions/Open Questions); re-plan if any spike failed its budget
10
+
11
+
## 2. Block Graph Core (trawler-core)
12
+
13
+
- [ ] 2.1 Define graph types: node IDs, block/page/tag/date nodes, property values (text/number/date/bool/ref)
14
+
- [ ] 2.2 Loro storage layer: movable tree + text + metadata maps; open/create graph directory; snapshot + update-blob persistence with crash-safe write ordering
15
+
- [ ] 2.3 Outline operations: create/delete/split/merge blocks, move subtree, reorder siblings — as transactions over the Loro doc
16
+
- [ ] 2.4 Inline reference model: parse refs/tags/dates out of block Markdown on commit (skipping code spans/fences); represent as typed spans
17
+
- [ ] 2.5 Derived in-memory graph: adjacency, backlink sets, tag/property/date indexes; full rebuild from Loro doc
18
+
- [ ] 2.6 Incremental index maintenance on edit transactions
19
+
- [ ] 2.7 Golden test: randomized 1,000+ edit sequences — incremental indexes equal full rebuild (spec: block-graph/Incremental index maintenance)
20
+
- [ ] 2.8 Crash-safety test: kill after acknowledged edit; verify recovery (spec: block-graph/Loro document is the source of truth)
21
+
22
+
## 3. Search Core (trawler-core)
23
+
24
+
- [ ] 3.1 Tantivy index over block content; disposable index directory with rebuild-on-missing
25
+
- [ ] 3.2 Incremental index updates within 1s of edit (spec: search/Index freshness)
26
+
- [ ] 3.3 Ranked search API with match spans for highlighting
27
+
- [ ] 3.4 Similarity trait + lexical (more-like-this) implementation excluding self/ancestors/descendants
28
+
29
+
## 4. Steel Query Engine (trawler-core)
30
+
31
+
- [ ] 4.1 Embed Steel VM using the S2 containment pattern; read-only capability context with only graph primitives registered
32
+
- [ ] 4.2 Set-algebra primitives: tag/ref/property/date-range/descendants/search sets; native and/or/not over set handles
33
+
- [ ] 4.3 Per-block Scheme predicate application over narrowed sets
34
+
- [ ] 4.4 Result shapes: outline slice (block IDs) and table (projected properties); error type with position info
35
+
- [ ] 4.5 Performance test: intersect 20k-tag × 5k-date sets on 100k-block graph < 50ms (spec: steel-queries/Set-algebra query primitives)
36
+
- [ ] 4.6 Timeout/cancellation tests: non-terminating query terminates at budget, VM context is reusable or respawned cleanly
37
+
38
+
## 5. GPUI Shell & Outline Editor (trawler)
39
+
40
+
- [ ] 5.1 App shell: window, theme, keymap infrastructure, graph open/create flow
41
+
- [ ] 5.2 Virtualized outline view rendering block trees from trawler-core (spec: outline-editor/Large outlines render lazily)
42
+
- [ ] 5.3 Markdown renderer for inactive blocks (pulldown-cmark or tree-sitter-markdown): paragraphs, emphasis, code spans/fences, quotes, embedded lists, links, plus node-reference inline extension
43
+
- [ ] 5.4 One-hot editor: multi-line focused-block editor, focus model, editor attach/detach, commit-on-blur; measure transition latency from the first working build (spec: outline-editor/Editing latency)
44
+
- [ ] 5.5 Outline keybindings: Enter/new block, Shift+Enter/newline within block, split at cursor, Tab/Shift+Tab, move up/down, backspace-merge, fold/unfold
45
+
- [ ] 5.6 Reference completion: `[[` and `#` popup with fuzzy matching; date completion for journal refs
46
+
47
+
## 6. Journal & Navigation (trawler)
48
+
49
+
- [ ] 6.1 Journal timeline home view: today's page auto-created and focused, prior non-empty days lazy-load below
50
+
- [ ] 6.2 Date rollover handling while app is running; empty-page suppression
51
+
- [ ] 6.3 Follow-reference keyboard action; navigation history with back/forward restoring scroll+focus
52
+
- [ ] 6.4 Backlinks section on every page/zoom view, grouped by source page, navigable
53
+
- [ ] 6.5 Block zoom with ancestor breadcrumb; zoom-out
54
+
- [ ] 6.6 Quick open: fuzzy switcher over pages/tags/dates
55
+
- [ ] 6.7 Search UI: invoke, ranked results with highlights, jump to block
56
+
- [ ] 6.8 Similar-blocks surface for the focused block (placement per prototype feedback; update design.md Open Questions with the outcome)
57
+
58
+
## 7. Query Blocks UI (trawler)
59
+
60
+
- [ ] 7.1 Query block type: authoring affordance, persistence as ordinary block with query flag
61
+
- [ ] 7.2 Scheme editing in the one-hot editor: tree-sitter highlighting, matching-paren indication
62
+
- [ ] 7.3 Debounced background evaluation wired to the query engine; inline error display
63
+
- [ ] 7.4 Result rendering: outline slice with navigation; table view with sortable columns for projected properties
64
+
- [ ] 7.5 End-to-end latency check: edit query → preview update feels live on a realistic graph
65
+
66
+
## 8. Hardening & Wrap-up
67
+
68
+
- [ ] 8.1 Dogfood checkpoint: use trawler as the daily driver for one week; file and triage findings
69
+
- [ ] 8.2 Performance pass against spec budgets (editing latency, query latency, index freshness, cold start)
70
+
- [ ] 8.3 Documentation: README (build/run), graph directory format notes, query language reference with examples
71
+
- [ ] 8.4 Reconcile artifacts: update design.md decisions/open questions to match reality; verify specs against implementation