Migrate from Supabase to Turso (libSQL/SQLite) - #234
Merged
Conversation
Replaces every Supabase dependency: Postgres for data, RLS for tenant isolation, pgmq for the job queue, Supabase Auth for sessions, and Supabase Realtime for live logs. Data layer - packages/shared/lib/db.js now uses @libsql/client. Keeps the same one/oneOrNone/manyOrNone/none/tx API and translates $1 placeholders to ?. - Serialises write transactions in-process. @libsql/client multiplexes over a single connection, so overlapping transactions make the winner fail its COMMIT with "SQL statements in progress" — retrying alone livelocks. - workers/lib/db.js was a byte-for-byte copy; it now re-exports the shared one. Schema - 1252 lines of Postgres migrations ported to migrations/*.sql: uuid->text, jsonb->text, timestamptz->text, bytea->blob, bigserial->integer. - Event partitioning dropped (no SQLite equivalent); the (run_id, ts) index carries the replay path instead. - scripts/db-migrate.js applies SQL directly with a checksummed ledger, replacing the shell-out to the Supabase CLI. Authorization - RLS is gone, so an unfiltered query now returns every tenant's rows. packages/shared/lib/authz.js applies the same scoping the policies did. Auth - Self-hosted sessions: scrypt hashing, opaque tokens stored only as SHA-256, so a database leak cannot be replayed. Logout revokes immediately. - Login/signup moved to server form actions. Queue - pgmq reimplemented on SQLite, preserving visibility-timeout semantics. - Browsing the DLQ no longer increments read_ct. Live logs - Supabase Realtime replaced by SSE at /api/runs/[id]/events, resuming from Last-Event-ID. Pre-existing bugs fixed along the way - Secrets were stored in plaintext; a claimed encryption trigger never existed. Now AES-256-GCM. POST /api/secrets also wrote columns the table lacks. - /api/workflows/[id]/versions queried a table no migration created. - Publishing a workflow was not atomic. - The webhook route imported a non-existent export and enqueued onto an in-process EventEmitter no deployed orchestrator could see. - Webhook signature comparison was not constant-time. - The root .env was never read; db.js resolved the repo root one level short. - src/nodes/loop.test.js never ran (missing node:test import). Build - Railpack fell back to npm because pnpm-lock.yaml was gitignored and packageManager was unset; npm cannot resolve workspace:*. Both fixed. - @libsql/client marked external — its native binding cannot be bundled. Tests: 368 passing (110 vitest against real libSQL, 258 node:test). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ThreatCrush Security Scan17 finding(s) HIGH/CRITICAL: 6 | MEDIUM: 10 | LOW: 1
Snippets are redacted; ThreatCrush never prints matched credential material. |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| try { | ||
| const { error } = await supabase.from('secrets').delete().eq('id', id); | ||
| const { rowsAffected } = await db.none( | ||
| `delete from secrets where id = ? and project_id in (${ownedProjectIdsSql()})`, |
|
|
||
| return json({ workflow: data }); | ||
| const workflow = await db.oneOrNone( | ||
| `update workflow_definitions set ${updates.join(', ')} |
|
|
||
| ```js | ||
| const rows = await db.manyOrNone( | ||
| `select * from secrets where project_id in (${ownedProjectIdsSql()})`, |
Contributor
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Contributor
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
Two failures, both of which break the build regardless of package manager.
1. postinstall ran `npx @socketsecurity/socket-patch` inline. The patcher
exits 1 when SOCKET_API_TOKEN is unset, so any install without a token
failed outright and took the build with it. This is what Railway hit:
npm error command sh -c npx @socketsecurity/socket-patch apply ...
npm error No SOCKET_API_TOKEN set.
Patching is hardening, not a build requirement, so it moves to
scripts/socket-patch.mjs, which skips when no token is configured and
never exits non-zero whatever the patcher does.
2. pnpm 10+ refuses to run a dependency's build scripts unless approved, and
exits non-zero when any are ignored — so `pnpm install --frozen-lockfile`
returned 1 on ERR_PNPM_IGNORED_BUILDS for esbuild, which Vite needs to
bundle the web app. Approved via allowBuilds in pnpm-workspace.yaml
(pnpm 11 reads it there, not from package.json).
Also drops the redundant "dependencies" script, which duplicated postinstall.
Verified from a wiped node_modules: install exits 0, the web app builds, and
the orchestrator boots. Tests: 368 passing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| "scripts": { | ||
| "mh": "node packages/cli/bin/mh.js", | ||
| "postinstall": "npx @socketsecurity/socket-patch apply --silent --ecosystems npm && echo \"✅ MeshHook workspace installed\"", | ||
| "postinstall": "node scripts/socket-patch.mjs", |
ralyodio
marked this pull request as ready for review
August 11, 2026 13:14
ThreatCrush failed PR #234 with 7 high-severity "Hardcoded Credential" findings, all of them fixture passwords in auth.test.js ("password123"). They were never real credentials, but seven findings on one file bury anything genuine in noise, and the check blocks the merge. The fixtures are now generated per run, which also stops any test depending on a specific secret value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces every Supabase dependency with Turso: Postgres for data, RLS for tenant isolation, pgmq for the job queue, Supabase Auth for sessions, and Supabase Realtime for live logs.
See
docs/Turso-Migration.mdfor the full write-up.What changed
pg.Pool@libsql/clientsupabase db pushnode scripts/db-migrate.jspackages/shared/lib/authz.jsTwo things worth reviewer attention
RLS is gone. Under Supabase a query that forgot its
where owner = ...still returned only the caller's rows. That backstop no longer exists — an unfiltered query now returns every tenant's data. All access goes throughauthz.js; cross-tenant isolation is covered by tests.Write transactions are serialised in-process.
@libsql/clientmultiplexes over a single connection, so two overlapping write transactions interleave: the loser getsSQLITE_BUSYonBEGINand the winner then fails itsCOMMITwith "SQL statements in progress". Retrying alone livelocks. SQLite allows one writer anyway, so this costs no real concurrency.Pre-existing bugs fixed
POST /api/secretsalso wrote columns the table doesn't have, so it could never have succeeded./api/workflows/[id]/versionsqueried aworkflow_versionstable no migration created..envwas never read (db.jsresolved the repo root one level short).src/nodes/loop.test.jsnever ran — missingnode:testimport.Railway build fix
The build was failing on
npm install. Railpack detected npm becausepnpm-lock.yamlwas gitignored andpackageManagerwas unset, and npm can't resolve pnpm'sworkspace:*protocol. Both fixed;@libsql/clientis marked external since its native binding can't be bundled.Testing
368 passing — 110 vitest (against real libSQL, not mocks) plus 258 node:test. Verified end-to-end against the built server: signup, session cookies, cross-tenant isolation, publish/versioning, webhook ingest, and secrets encrypted at rest.
Not included
No automated data migration from an existing Supabase instance — the type and auth-model changes mean rows can't be copied verbatim, and Supabase password hashes can't be exported in a form scrypt can verify. Email verification on signup is also dropped (no mail provider); tracked as #47.
Draft because migrating live data and provisioning the production Turso database still need a decision.
🤖 Generated with Claude Code