Skip to content

Feature: Interactive Shell 💻#508

Open
tnaum-ms wants to merge 329 commits intonextfrom
feature/shell-integration
Open

Feature: Interactive Shell 💻#508
tnaum-ms wants to merge 329 commits intonextfrom
feature/shell-integration

Conversation

@tnaum-ms
Copy link
Copy Markdown
Collaborator

@tnaum-ms tnaum-ms commented Feb 17, 2026

Shell Integration — DocumentDB Query Language & Autocomplete

Umbrella PR for the shell integration feature: a custom documentdb-query Monaco language with intelligent autocomplete, hover docs, and validation across all query editor surfaces (filter, project, sort, aggregation, query playground).

Work is organized as incremental steps, each delivered via a dedicated sub-PR merged into feature/shell-integration.


Progress

Post-Step 8 — UX Polish & Tree View Enhancements (direct pushes)

After merging Step 8 (Interactive Shell), the following UX improvements were pushed directly to the branch:

  • Inline action buttons in tree view — Added inline icon buttons to database and collection tree items across Connections, Discovery, and Azure views:

    • Database nodes: $(terminal) Open Interactive Shell and $(keyboard) New Query Playground inline buttons
    • Collection nodes: $(files) Open Collection, $(terminal) Open Interactive Shell, and $(keyboard) New Query Playground inline buttons
    • Updated command icons: shell → $(terminal), collection open → $(files), playground → $(keyboard)
  • Custom editor tab icons — Added dedicated SVG icons for webview editor tabs:

    • Collection View tabs now show a custom collection icon (light/dark variants) instead of the default webview icon
    • Document View tabs now show a custom document icon (light/dark variants)
    • Query Playground language definition now includes a custom keyboard icon (playground-light.svg / playground-dark.svg)
  • Double-click to open Collection View — The Documents tree item now requires a double-click to open the Collection View, preventing accidental tab opens on single-click browse. Implemented via a reusable registerDoubleClickCommand() utility (500ms debounce window). Tooltip updated to hint at the double-click behavior.

  • Published AI & dev documentation — Committed the full series of planning documents used during development (docs/ai-and-plans/interactive-shell/ and docs/ai-and-plans/future-work/), covering the high-level plan, all step-level design docs, and future work proposals.

Key Architecture Decisions

Decision Outcome
Language strategy documentdb-query custom language — JS Monarch tokenizer, no TS worker (~400-600 KB saved)
Completion providers Single CompletionItemProvider + URI routing (documentdb://{editorType}/{sessionId})
Completion data documentdb-constants bundled at build time; field data pushed via tRPC subscription
Validation acorn.parseExpressionAt() for syntax errors; acorn-walk + documentdb-constants for identifier validation
Document editors Stay on language="json" with JSON Schema validation
Scratchpad language="documentdb-playground" referencing built-in JS grammar; in-process eval with @mongosh packages reusing existing MongoClient
Interactive Shell Pseudoterminal-based REPL with dedicated worker per session, persistent @mongosh context, CommandInterceptor for shell commands

@tnaum-ms tnaum-ms linked an issue Feb 17, 2026 that may be closed by this pull request
@tnaum-ms tnaum-ms added this to the 0.8.0 - February 2026 milestone Feb 17, 2026
@tnaum-ms tnaum-ms linked an issue Feb 17, 2026 that may be closed by this pull request
@tnaum-ms tnaum-ms changed the title feature: shell integration feature: shell integration 💻 Feb 23, 2026
@tnaum-ms tnaum-ms changed the title feature: shell integration 💻 [WIP] feature: shell integration 💻 Feb 25, 2026
tnaum-ms added 22 commits March 16, 2026 19:00
Replace format pattern 'yyyy-MM-ddTHH:mm:ssZ' with a valid example
'2025-01-01T00:00:00Z'. The format pattern was confusing — users
typed over it with values like '2222-11-11T11:11:111' which fail
ISODate validation.

Also update date range snippet with realistic start/end dates.
Previously `new Daddddte()` was not flagged because the validator only
handled CallExpression nodes. NewExpression has the same callee shape
but was not visited. Now unknown constructors in `new Foo()` produce
error diagnostics (red squiggles), and near-miss typos like `new Dae()`
produce warnings.
Add Date.now(), Math.floor(), Math.ceil(), Math.round(), Math.min(),
Math.max() as individual completion items alongside the class-level
Date/Math entries. Removed the bare `Math` entry since the individual
methods are more useful. All entries use sort prefix 4_ (after BSON
constructors at 3_).
…_COMPLETION_META

PROJECTION_COMPLETION_META was ['field:identifier'] which returned 0 static
entries — projection operators ($slice, $elemMatch, $) and BSON constructors
were never shown in project/sort editors. Added 'query:projection' and 'bson'
to the meta preset.
Never use git add -f to override .gitignore. Files in docs/plan/ and
docs/analysis/ are local planning documents excluded from the repo.
- Remove unnecessary escape in template literal (no-useless-escape)
- Formatter-applied changes to imports and whitespace
Tests were using 'int' but the schema analyzer produces 'int32' (BSONTypes.Int32).
Also fixed applicableBsonTypes in documentdb-constants bitwise operators to use
'int32' to match, which was a real production mismatch.
When cursorContext is undefined or 'unknown', show the full set of completions
(fields, all operators, BSON constructors, JS globals) instead of only key-position
items. This is a better UX fallback — when context can't be determined, the user
is better served by seeing everything available.
These debug logs were used during development and logged full editor text and
completion items to the webview console on every keystroke. Removed to avoid
unnecessary noise and potential data exposure.
Documentation links like [DocumentDB Docs](https://...) were not rendered as
clickable hyperlinks in Monaco's completion detail panel. Monaco requires
{ value: string, isTrusted: true } on MarkdownStrings to enable link rendering.

Set isTrusted: true on operator documentation MarkdownStrings in
mapOperatorToCompletionItem. This is safe because the documentation content
comes entirely from documentdb-constants (operator descriptions we control),
not from user-generated content.
@tnaum-ms tnaum-ms changed the title [WIP] feature: shell integration 💻 Feature: Interactive Shell 💻 Apr 10, 2026
Add informational messages when:
- Playground file is empty (runAll)
- No code to run / no code block at cursor (runSelected)
- No database connection (executePlaygroundCode)
- A playground is already running (executePlaygroundCode)

Previously these cases returned silently with no feedback.
@tnaum-ms
Copy link
Copy Markdown
Collaborator Author

C1 Fix: Silent Failures — Added user feedback for blocked operations

Multiple early-return paths in playground commands now show informational messages instead of silently returning:

  • Empty playground file → "The playground file is empty. Add some code to run."
  • No code to run / no code block at cursor → "No code to run. Select some code or place the cursor in a code block."
  • No database connection → "Connect to a database before running..."
  • Already executing → "A playground is already running. Wait for it to finish."

Commit: 9b956fb

Add initialization promise lock so concurrent evaluate() calls during
initialization await the same promise instead of spawning duplicate workers.
@tnaum-ms
Copy link
Copy Markdown
Collaborator Author

C2 Fix: Race condition — concurrent shell worker initialization

Added initialization promise lock in ShellSessionManager.evaluate(). Concurrent calls now await the same init promise instead of each spawning their own worker.

Commit: d7eecae

Wrap evaluateFresh() body in try/finally to call instanceState.close()
after each evaluation. Prevents leaking ShellInstanceState, ShellEvaluator,
and vm.Context resources on every playground run.
@tnaum-ms
Copy link
Copy Markdown
Collaborator Author

C3 Fix: Fresh mode resource leak

Wrapped evaluateFresh() body in try/finally to call instanceState.close() after each evaluation. Prevents leaking ShellInstanceState, ShellEvaluator, and vm.Context resources on every playground run.

Commit: 9f2a544

Add assertDocumentObject() guard after parseShellBSON() for projection
and sort parameters. Rejects scalars, arrays, and null with a clear
QueryError instead of passing invalid values to the driver.
@tnaum-ms
Copy link
Copy Markdown
Collaborator Author

H2 Fix: parseShellBSON type validation for projection/sort

Added assertDocumentObject() guard after parseShellBSON() in both runFindQuery() and streamDocumentsWithQuery(). Non-object values (scalars, arrays, null) now throw a clear QueryError instead of being silently cast to Document and producing confusing driver errors.

Commit: bc2fcd6

…rs (M5, M7, M8)

M5: Replace toLocaleString() with toISOString() for playground filenames
    to avoid locale-dependent characters in file paths.
M7: Replace localeCompare with sensitivity:'base' with strict === equality
    for query cache keys. Prevents false cache hits (e.g., café vs cafe).
M8: Log unexpected errors during AST walk instead of silently swallowing
    all exceptions. SyntaxErrors are still suppressed as expected.
@tnaum-ms
Copy link
Copy Markdown
Collaborator Author

M5, M7, M8 Fixes: Three medium-severity cleanups

  • M5: Replaced toLocaleString() with toISOString() for playground filenames to avoid locale-dependent characters (Arabic numerals, RTL marks)
  • M7: Replaced localeCompare with sensitivity: 'base' with strict === equality for query cache keys — café and cafe were incorrectly treated as the same query
  • M8: AST walk catch {} now logs unexpected errors via console.error while still suppressing SyntaxError as intended

Commit: d60288b

Show the human-readable clusterDisplayName when credential lookup fails
instead of the internal clusterId.
@tnaum-ms
Copy link
Copy Markdown
Collaborator Author

M6 Fix: Display name in credential error

Error message in PlaygroundEvaluator.buildInitMessage() now shows clusterDisplayName instead of the internal clusterId so users can identify which connection failed.

Commit: d1ad9b1

… M4)

M3: Add 300ms debounce to onDidChangeTextDocument handler in
    PlaygroundDiagnostics to avoid O(n) regex scans on every keystroke.
M4: Clear existing schema data before scanning in scanCollectionSchema()
    so users get a fresh scan result instead of accumulated stale entries.
@tnaum-ms
Copy link
Copy Markdown
Collaborator Author

M3, M4 Fixes: Diagnostics debounce & schema scan cleanup

  • M3: Added 300ms debounce to PlaygroundDiagnostics.onDidChangeTextDocument handler. Previously, every keystroke triggered a full-document regex scan with no throttling.
  • M4: scanCollectionSchema() now clears existing schema data for the collection before adding new samples, giving users a predictable "fresh scan" experience.

Commit: 20f608c

Replace four separate nullable fields (_persistentInstanceState,
_persistentEvaluator, _persistentContext, _persistentVmContext) and
_persistentInitialized flag with a single _persistent compound object.
Eliminates partial initialization risk and removes all non-null assertions.

Also fix test mock to include close() method for C3 compatibility.
@tnaum-ms
Copy link
Copy Markdown
Collaborator Author

M9 Fix: Consolidate persistent shell state into single compound object

Commit: c6d0558

Replace space-only word delimiter with alphanumeric+underscore+dollar
pattern for word navigation. Now db.collection.find() navigates by
word boundaries (db, collection, find) instead of requiring one space
per word.
@tnaum-ms
Copy link
Copy Markdown
Collaborator Author

M12 Fix: Word navigation uses proper word boundaries

wordLeft() and wordRight() in ShellInputHandler now use an alphanumeric/underscore/dollar pattern instead of space-only delimiters. Navigating db.collection.find() with Ctrl+Left/Right now stops at each identifier boundary instead of the next space.

Commit: 2a0f689

…(H4)

dropCollection/createCollection now clear the collections cache for
that database. dropDatabase/createDatabase clear the databases cache.
Prevents stale entries in autocomplete and tree views after mutations.
@tnaum-ms
Copy link
Copy Markdown
Collaborator Author

H4 Fix: ClustersClient cache invalidation after CRUD operations

dropCollection/createCollection now invalidate the collections cache for the affected database. dropDatabase/createDatabase invalidate the databases cache. This prevents stale entries in autocomplete and tree views after mutations.

Commit: 78d719f

Add documentDB.shell.initTimeout setting (default: 60 seconds) to prevent
the shell from hanging indefinitely if worker spawn or connection fails.
Wraps ensureWorker() in Promise.race against a timeout.
@tnaum-ms
Copy link
Copy Markdown
Collaborator Author

M2 Fix: Configurable shell initialization timeout

Added documentDB.shell.initTimeout setting (default: 60 seconds, minimum: 5). Shell initialization now uses Promise.race() against a timeout to prevent indefinite hangs if worker spawn or connection fails.

Commit: 8c3b760

- Update l10n bundle with new user-facing strings from review fixes
- Fix no-unsafe-assignment lint errors in ClustersClient parseShellBSON
- Apply prettier formatting to newPlayground.ts and runSelected.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve Scrapbook Experience (shell integration) 🚀

3 participants