Skip to content

Try running PHPUnit tests against Doltlite#371

Draft
JanJakes wants to merge 20 commits intotrunkfrom
doltlite
Draft

Try running PHPUnit tests against Doltlite#371
JanJakes wants to merge 20 commits intotrunkfrom
doltlite

Conversation

@JanJakes
Copy link
Copy Markdown
Member

Summary

  • Adds a new PHPUnit Tests (Doltlite) GitHub Actions job.
  • The job builds Doltlite from source as a shared library and runs the packages/mysql-on-sqlite PHPUnit suite with LD_PRELOAD=libdoltlite.so, so PHP's PDO SQLite transparently uses Doltlite's prolly-tree storage engine instead of stock SQLite.
  • Intended as an exploratory baseline — we want to see how the existing test suite behaves when the underlying storage engine is swapped.

Notes

  • Doltlite advertises itself as a drop-in SQLite replacement (same sqlite3.h API, same shared library symbols). LD_PRELOAD is the same approach used in Doltlite's own Python example.
  • A debugging step prints the pdo_sqlite.so ldd output, SELECT SQLITE_VERSION(), and SELECT doltlite_engine() (should be prolly) before the test run to confirm the preload actually took effect.
  • The workflow currently pins to PHP 8.3 on ubuntu-latest. Doltlite is based on SQLite 3.53.0, well above the driver's MINIMUM_SQLITE_VERSION of 3.37.0.

Test plan

  • PHPUnit Tests (Doltlite) job runs on this draft PR.
  • The verification step shows SELECT SQLITE_VERSION() returning 3.53.0 and doltlite_engine() returning prolly.
  • Review the PHPUnit output to see which tests (if any) fail under Doltlite.

JanJakes added 20 commits April 24, 2026 10:12
Build Doltlite from source and run the mysql-on-sqlite PHPUnit suite
against it via LD_PRELOAD, so PHP's PDO SQLite transparently uses the
Doltlite engine.

See: https://github.com/timsehn/doltlite
The first attempt used LD_PRELOAD, but pdo_sqlite.so's DT_NEEDED on
libsqlite3.so.0 resolved to the system SQLite and the preload was
ignored — the tests passed against stock SQLite, not Doltlite.

Physically replace /usr/local/lib/libsqlite3.so.0 (and the multiarch
symlink) with libdoltlite.so, run ldconfig, and assert that
doltlite_engine() returns 'prolly' before running PHPUnit.
pdo_sqlite references sqlite3_column_table_name, which is gated behind
SQLITE_ENABLE_COLUMN_METADATA. Match the CFLAGS used by the stock-SQLite
phpunit workflow so pdo_sqlite can bind against the Doltlite build.
The canonical repo is dolthub/doltlite (SQLite 3.54.0 base, actively
developed), not the personal mirror at timsehn/doltlite which trails
behind.
Doltlite auto-converts tables with composite/TEXT primary keys to
WITHOUT ROWID, which strips the ROWID column and breaks the driver's
ORDER BY ROWID tie-breaks in info-schema reads. The auto-conversion
is a single tabOpts |= TF_WithoutRowid in src/build.c.

Sed that line to a no-op before building so composite-PK tables take
SQLite's normal rowid + unique-PK-index path, and assert at runtime
that a composite-PK test table exposes a rowid.
Seven minimal PHP/PDO probes to narrow where the remaining ALTER TABLE
failures come from: sequential DELETEs on rowid and composite-PK tables,
prepared-statement reuse, INSERT+DELETE ordering, NOCASE round-trip,
and the INSERT..SELECT table-rebuild path.
The NOCASE-fold path in src/sortkey.c is implicated in the transaction
write-visibility bug the probes isolated. Falling through to the BINARY
path skips the fold preprocessing while leaving SQLite's collation logic
at the SQL layer intact. Driver usage never inserts case-differing
values at the same unique-index position, so the uniqueness semantics
loss is not hit.
The 8 failing tests all hit the same Doltlite transaction-visibility
bug on the rowid-patched storage path (two sequential writes inside
a SAVEPOINT against a composite NOCASE PK lose the second write).
The NOCASE-to-BINARY sortkey patch I tried made things slightly
worse, so the bug is deeper in the transaction path. Skip the
eight tests via a negative-lookahead phpunit --filter so the job
stays green until the upstream bug is fixed, and drop the now-done
investigation probes.
Root cause of the ALTER TABLE failures: in prollyBtCursorIndexMoveto,
when the cursor has a non-empty mutmap, findMatchingMutMapEntry gets
called after the tree scan. Its internal sqlite3VdbeRecordCompare
probes reset pIdxKey->eqSeen to 0 on each call, clobbering the flag
that the tree scan loop set to 1 for a prefix match. OP_SeekGE with
BTREE_SEEK_EQ then sees eqSeen=0 and bails to seek_not_found,
silently skipping the DELETE. The fix saves eqSeen before
findMatchingMutMapEntry and restores it whenever the mutmap branch
isn't taken.

Reproduced and traced to this exact line via source-level printf in a
local Doltlite build; the fix makes the cross-check in the verify
step (two sequential DELETEs inside a SAVEPOINT on a composite
NOCASE PK) return the expected single remaining row.

Drop the previous test exclusions; run the full PHPUnit suite.
Switch from inline heredoc (broke YAML parsing — unindented patch
content ended the run block early) to a checked-in unified diff
applied with plain patch -p1. Clearer, reviewable, and the diff file
can be sent upstream as-is.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant