Add PHPUnit tests workflow for Turso DB by JanJakes · Pull Request #370 · WordPress/sqlite-database-integration

JanJakes · 2026-04-23T14:23:43Z

Summary

Adds a CI job that runs the SQLite driver unit tests (packages/mysql-on-sqlite) against Turso DB, a Rust reimplementation of SQLite.

The workflow:

Clones Turso at its latest release tag.
Builds the turso_sqlite3 crate as a cdylib (a drop-in for libsqlite3's C API).
Preloads it via LD_PRELOAD so PHP's pdo_sqlite resolves its sqlite3_* symbols against Turso instead of the system libsqlite3.
Runs PHPUnit.

The job is informational: Turso is in beta with a partially implemented SQLite C API, so failing tests are expected. The step uses continue-on-error: true so the job still succeeds and the test output is visible; we can track compatibility progress over time.

Refs: #204

Test plan

CI workflow "PHPUnit Tests (Turso DB)" runs on this PR.
turso_sqlite3 builds successfully from source.
The "Verify Turso shared library exposes SQLite3 C API" step prints sqlite3_* symbols.
The "Report SQLite version via Turso preload" step prints a version string (confirming LD_PRELOAD is wired up correctly).
PHPUnit runs to completion and prints a pass/fail summary.

Adds a CI job that runs the SQLite driver unit tests against Turso DB (https://github.com/tursodatabase/turso), a Rust reimplementation of SQLite. The workflow builds the turso_sqlite3 crate as a cdylib and preloads it via LD_PRELOAD so PHP's pdo_sqlite resolves its sqlite3_* symbols against Turso instead of the system libsqlite3. The job is informational: Turso is in beta with a partially implemented SQLite C API, so failing tests are expected. Refs: #204

Turso v0.5.3 does not implement sqlite3_set_authorizer, which pdo_sqlite calls during PDO construction, so opening a SQLite PDO connection panics the Turso library. Mark the diagnostic step as continue-on-error so the rest of the job still runs and produces a PHPUnit report.

Turso's stub!() macro expands to todo!() and aborts the PHP process whenever pdo_sqlite calls an unimplemented sqlite3_* function. The first one hit is sqlite3_set_authorizer during PDO construction, so the tests can't even start. Rewrite the macro via sed before building so stubbed functions return a zeroed value of their return type (0 / SQLITE_OK for ints, NULL for pointers). This lets the driver proceed past optional calls and exercise parts of Turso that are actually implemented.

PHPUnit currently segfaults right after printing its banner, before any test runs. Replace the single sqlite_version() check with a script that exercises PDO basics (create table, insert with exec and with prepared statement, select, destruction) so CI logs show which operation Turso can't handle.

The diagnostics have served their purpose: phpunit --version, --list-tests, and standalone bootstrap all run fine, so the segfault is specific to test execution (likely inside WP_SQLite_Driver's setUp path). Drop the step to keep the workflow focused on the pass/fail signal.

The driver's translator emits e.g. THROW(...) in uppercase, but the UDFs are registered with lowercase names like 'throw'. Turso's extension registry stores the name as-is and connection.rs does a case-sensitive HashMap lookup, so 32 tests fail with 'no such function: THROW'. Normalise to lowercase at both register and lookup sites.

On Turso main the query planner has matured; try again to rewrite the row-value UPDATE into two correlated subqueries. This would fix the 82 tests (61 direct + 21 wp_die-wrapped) failing on '2 columns assigned 1 values'.

Turso's aggregate-without-GROUP-BY returns 0 rows when the WHERE clause matches nothing (SQL standard requires 1 row with NULL aggregates). Zero rows from a scalar subquery yields NULL, which violates the NOT NULL constraint on is_nullable. Wrap each subquery in IFNULL(..., c.<col>) so no-match cases keep the existing value.

Turso prefixes constraint errors with 'Runtime error: ' and appends the raw SQLite error code. Tests compare against SQLite's native format, so normalise at rethrow time.

Turso's sqlite3_create_function_v2 has 32 pre-generated bridge trampolines (MAX_CUSTOM_FUNCS). The driver registers 44 UDFs, so the last 12 silently fail registration; SQL calls to from_base64 etc. then fail with 'no such function'. Generate 32 more bridges.

The create_function_v2 reuse-by-name branch invokes the previous slot's destroy callback with its stored p_app. In PHPUnit usage (setUp opens a fresh PDO per test), by the time a second setUp re-registers a same-named UDF, the previous PDO is already gone and its destroy callback UAFs, which hangs pdo_sqlite indefinitely (this is what was hanging testFromBase64Function, testAlterTableAdd*, etc.). Drop the destroy invocation — callbacks still fire from the PHP side at real PDO-destruction time.

TextValue stores Box<str>::into_raw() cast to *const u8 (losing the length from the fat pointer). free() reconstructs Box<u8> (size 1 byte) and frees what the allocator tracks as a larger allocation, corrupting the heap. This was the segfault in UDF result freeing (hitting every test that returns text/blob from a PHP UDF, including testFromBase64Function). Rebuild the fat pointer from the stored length at free time.

Bisect: testReconstructTable hangs for the full 10-min budget once Translation_Tests run before it. With Translation_Tests skipped (prior state at a618745), reconstructor tests complete in ~0.3 s. - Skip WP_SQLite_Driver_Translation_Tests in the main run to unblock the workflow. Lose 56 tests temporarily. - Add two probes: 1) testReconstructTable in isolation (expect pass). 2) Translation_Tests + testReconstructTable (expect hang, confirms). The probes give us a signal for what's actually corrupting Turso state so we can target the fix instead of guessing.

Isolate which test class before Translation_Tests leaves Turso in a state that makes testReconstructTable hang in the main run. Probes run three combinations: Driver_Tests, Metadata_Tests, PDO_API_Tests — each paired with Translation_Tests and testReconstructTable. Whichever probe hangs identifies the polluting suite.

All subset probes (Driver+Translation+Reconstruct at 400 tests, Metadata+Translation+Reconstruct, PDO_API+Translation+Reconstruct) complete with testReconstructTable passing. Only the full main run with Translation_Tests unskipped triggers the 10-min hang. Add a reproduction probe that runs the main filter with Translation unskipped, with a watchdog that attaches gdb at 150s (before the 180s timeout) to dump thread backtraces of the hanging PHP process.

…hdog Previous attempt exhausted the 30-min job budget on subset probes. All subsets pass — only the full main run with Translation_Tests unskipped reproduces the hang, confirmed at testReconstructTable start. Drop the subset probes. Keep one probe that reproduces 4c4f491's main-run state with: - longer timeout (420s) so we sit *in* the hang, not at its start - pgrep -x php to target PHP itself (not the timeout wrapper we captured last time) - /proc/<pid>/stack + /proc/<pid>/wchan for kernel-side picture - two gdb snapshots (T+360s, T+400s) in case one detaches early

Root cause captured by gdb: during a UDF callback fired from sqlite3_step (which holds db.inner mutex), PHP's cycle GC fires a PDO statement destructor. The destructor calls sqlite3_finalize, which blocks on the same non-reentrant std::sync::Mutex. #5 sqlite3_finalize <- waits on db.inner #7 php_pdo_free_statement #12 zend_gc_collect_cycles #13-39 execute_ex <- UDF callback running user PHP Patch Turso's sqlite3_finalize and stmt_run_to_completion to use try_lock. On contention (re-entrant path), skip the drain and the stmt_list unlink. The list is only walked by sqlite3_next_stmt (unused by pdo_sqlite) and dropped on close, so a stale entry is harmless; the stmt Box is still freed. Unskip Translation_Tests in the main run now that the hang is fixed.

JanJakes added 3 commits April 23, 2026 16:23

Trigger CI re-run

47c9341

JanJakes closed this Apr 23, 2026