Skip to content

Add pseudo listen/notify mechanism for SQLite#1275

Draft
brandur wants to merge 1 commit into
masterfrom
brandur-sqlite-listen-notify
Draft

Add pseudo listen/notify mechanism for SQLite#1275
brandur wants to merge 1 commit into
masterfrom
brandur-sqlite-listen-notify

Conversation

@brandur
Copy link
Copy Markdown
Contributor

@brandur brandur commented Jun 5, 2026

A fairly long-standing issue with SQLite compared to Postgres is the
lack of a listen/notify system because SQLite doesn't offer such a
mechanism. This has forced SQLite to operate in poll-only mode which
means there's a longer delay before it sees jobs that it can work.

Honker [1] hit Hacker News a few weeks/months back. It implements a
listen/notify-like mechanism, and when reading about how it's
implemented, I realized it wouldn't be all that crazy to do something
similar for River. Basically, it's poll based, and relies on polling a
single table fairly actively. That's something that's not great in
general, but we already do that for poll-only, except that it's multiple
tables instead of just one (i.e. river_job + river_leader).

Here, I propose that we implement listen/notify using a similar strategy
by adding a river_notification in SQLite. Like listen/notify, this
table has a listen/notify, along with a timestamp that we can use to
track event creation and prune older rows. We add a SQLite Listener
similar to the one for Postgres, but which polls river_notification,
remembering the last ID that it so it can make sure to only fetch new
notifications in each River process.

A new maintenance service that runs only in SQLite takes care of pruning
events in the table older than 5 minutes.

A nice property of river_notification is that like listen/notify, it
all works transactionally as well. New rows are hidden until the
transaction that inserted them commits, keeping them invisible from the
listener. After commit, the listener sees them and responds. We track a
lastID in the listener for more efficient reads, which seems like it
might be a problem due to out-of-order IDs, but for better or worse this
is not possible in SQLite due its single-writer model. IDs always commit
in order.

You might think that this would create a table that's overly high volume
because 1,000 job insertions would create 1,000 notifications, but
luckily we've been deduplicating notifications through the use of a
notify limiter (i.e. insertNotifyLimiter.ShouldTrigger(queue)) for a
long time now. Leadership notifications are also low volume, so this
table should never have to be very big at any given time, which is nice.

Requires a database migration, but that might dovetail fairly nicely
with #1224 (convert SQLite json to jsonb), which also does.

I've experimented with a few versions of this. This one also adds a
river_notification to Postgres, though doesn't use it. The rationale
here is (1) that the Postgres river_notification makes it easier to
test the client as if it was SQLite but without having to bring a SQLite
driver into the top-level project as a dependency, and (2) it may be a
useful alternative to poll-only even in Postgres down the line, where
something like a bouncer makes listen/notify difficult to use.

[1] https://github.com/russellromney/honker

@brandur brandur force-pushed the brandur-sqlite-listen-notify branch from 37fe29c to 44e8909 Compare June 5, 2026 15:04
@brandur brandur marked this pull request as draft June 5, 2026 15:06
@brandur brandur force-pushed the brandur-sqlite-listen-notify branch 2 times, most recently from 1f2087b to 775c6f0 Compare June 5, 2026 15:47
A fairly long-standing issue with SQLite compared to Postgres is the
lack of a listen/notify system because SQLite doesn't offer such a
mechanism. This has forced SQLite to operate in poll-only mode which
means there's a longer delay before it sees jobs that it can work.

Honker [1] hit Hacker News a few weeks/months back. It implements a
listen/notify-like mechanism, and when reading about how it's
implemented, I realized it wouldn't be all that crazy to do something
similar for River. Basically, it's poll based, and relies on polling a
single table fairly actively. That's something that's not great in
general, but we already do that for poll-only, except that it's multiple
tables instead of just one (i.e. `river_job` + `river_leader`).

Here, I propose that we implement listen/notify using a similar strategy
by adding a `river_notification` in SQLite. Like listen/notify, this
table has a listen/notify, along with a timestamp that we can use to
track event creation and prune older rows. We add a SQLite `Listener`
similar to the one for Postgres, but which polls `river_notification`,
remembering the last ID that it so it can make sure to only fetch new
notifications in each River process.

A new maintenance service that runs only in SQLite takes care of pruning
events in the table older than 5 minutes.

A nice property of `river_notification` is that like listen/notify, it
all works transactionally as well. New rows are hidden until the
transaction that inserted them commits, keeping them invisible from the
listener. After commit, the listener sees them and responds. We track a
`lastID` in the listener for more efficient reads, which seems like it
might be a problem due to out-of-order IDs, but for better or worse this
is not possible in SQLite due its single-writer model. IDs always commit
in order.

You might think that this would create a table that's overly high volume
because 1,000 job insertions would create 1,000 notifications, but
luckily we've been deduplicating notifications through the use of a
notify limiter (i.e. `insertNotifyLimiter.ShouldTrigger(queue)`) for a
long time now. Leadership notifications are also low volume, so this
table should never have to be very big at any given time, which is nice.

Requires a database migration, but that might dovetail fairly nicely
with #1224 (convert SQLite json to jsonb), which also does.

I've experimented with a few versions of this. This one also adds a
`river_notification` to Postgres, though doesn't use it. The rationale
here is (1) that the Postgres `river_notification` makes it easier to
test the client as if it was SQLite but without having to bring a SQLite
driver into the top-level project as a dependency, and (2) it may be a
useful alternative to poll-only even in Postgres down the line, where
something like a bouncer makes listen/notify difficult to use.

[1] https://github.com/russellromney/honker
@brandur brandur force-pushed the brandur-sqlite-listen-notify branch from 775c6f0 to dfd504b Compare June 5, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant