Zero 1.11

Cached Results and Sturdier Client Storage

Installation

Copynpm install @rocicorp/zero@1.11

Overview

Zero 1.11 shows cached query results sooner, recovers from client storage failures, and makes Litestream v5 the default for backups. It also speeds up queries on the client and server and fixes a long list of correctness and reliability issues.

Features

• Queries whose results a previous session stored locally now start with the new 'cached' result type, so apps can show stored data right away and still tell it apart from data confirmed by the server. (#6578) (thanks @pawarren!)
• Building the same query again usually returns the same Query object, so useQuery no longer rebuilds and rehashes its query on every render, and queries built inline can be used as React dependencies. (#6469, #6496) (thanks @charpeni!)
• Zero now starts in a new 'initializing' connection state while it loads local data. Slow local loading no longer counts against the connect timeout. (#6561)
• When local storage fails to open, fills up, or reports I/O errors, Zero reports it once through the new onStorageFailure option and keeps running in memory. zero.kvStore reports which store is in use. (#6585, #6657, #6686) (thanks @pawarren and @Karavil!)
• Upgrading from 1.10 keeps each client's local data and its pending mutations. Future protocol changes no longer start clients over with an empty local database. (d597c69e8, 28e581483)
• Server mutators can return a JSON value to the client, run again in a new transaction when they fail with an error that tx.retryOn() accepts, and run at a chosen transaction isolation level. (7756eb9bb, 4fe91c0d8, cb2821f3a) (thanks @pawarren!)
• Litestream v5 backups are now the default. Replicas restored from another replica's backup continue it incrementally, periodic snapshots keep backups compact, and the replica's WAL stays bounded. (fb536f686, c1a586810, #6476, #6539, #6717)
• A zero-cache that comes online now waits until replication has caught up before reporting ready. (1c5ddcdb4)
• Inactive client groups are now kept for 7 days instead of 48 hours, so clients that return within a week avoid a full resync. (#6484)
• New tools for diagnosing slow queries: zero-cache logs which queries are slow and why, with index suggestions, and periodically logs per-query statistics. A new operator password gives access to CPU profiles (/profz) and replica planner statistics (/plannerz) without the admin password. (#6446, #6639, #6673, #6675, #6699)
• Opt-in limits protect zero-cache from expensive queries: a hydration budget for reconnecting clients and a per-query hydration timeout. (#6477, #6499)
• New OpenTelemetry metrics for WAL usage, index-build progress in replication status events, and restore progress in a new view-syncer status event. (#6591, #6621, #6714)
• Server mutations can send their logs to your own LogSink. (#6677) (thanks @pawarren!)
• The zero Docker image shrank from about 1.09 GB to 807 MB. (#6695)

Performance

Server-Side Incremental Updates

When a transaction replicates, zero-cache brings every client group's queries up to date with it. Zero 1.11 does this about 2.5x faster: with 100 client groups on one sync worker, each syncing a query with a related subquery, advancing through a transaction takes about 2.5x less CPU.

Client Cold Start

Apps usually register their queries before Zero has loaded local data, for example when useQuery runs during startup. Zero 1.10 then pushed every loaded row through every registered query. Zero 1.11 waits until the data has loaded and hydrates each query once (#6472). With 13 queries and about 30,000 local rows, startup is about 7x faster.

Building and Hashing Queries

Components build their queries on every render, and Zero hashes each query to identify it. Building the same query now returns the same object with its hash already computed (#6469), so building and hashing a query is about 7x to 17x faster, depending on its size.

Slower in Zero 1.11: Client Writes With Many Joined Queries

One benchmark got slower. With 300 open queries, each joining the same four tables, applying one write in the client's query engine takes about 1.2x longer in Zero 1.11. Joins now keep an index of their parent rows so that changes no parent can see are skipped (#6645, #6708). When every change reaches some parent, as here, keeping that index up to date is extra work.

Fixes

Query Results

• Queries using or with a flipped exists no longer return wrong rows. (#6636)
• Joins no longer leave a deleted row on clients, keep a stale row in a limit() query, or fail with node does not exist. (#6660)
• Rows behind an exists inside a scalar subquery are now synced, and changes to them propagate. (913c0665f)
• Catching up a client after a large update no longer fails with Missing row, and a schema change during rehydration no longer fails with table has no column named. (#6662, #6616)
• Queries whose SQLite plan nests one multi-index OR inside another no longer crash zero-cache in a loop. (6ea17bfcd) (thanks @robertoborgone!)
• zero-cache no longer calls ZERO_QUERY_URL again for unchanged custom queries when it rebuilds its query pipelines. Permission changes still re-transform every query. (#6601)
• In React and SolidJS, two custom queries with the same query but different names or arguments no longer share one view. A SolidJS view rebuilt after its query changes uses the current TTL. (#6461, #6497)
• In mutators, a row just inserted without its optional columns now reads back with those columns as null. (d4816752c)

Clients and React Native

• SQLite-backed stores (expo-sqlite, op-sqlite) no longer return corrupted data under concurrent reads, hang after a failed transaction, or keep part of a failed commit. (#6495, #6655, #6683) (thanks @pawarren!)
• Healthy clients no longer log ChunkNotFoundError during persist, and a write that would corrupt the local store now fails instead. (#6670, #6559) (thanks @pawarren!)
• With op-sqlite and a custom location or encryptionKey, dropping the database now deletes the right file. (#6656) (thanks @pawarren!)
• Clients no longer go offline after the app resumes from the background or the computer wakes from sleep, and React Native connect timeouts retry instead of leaving the client stuck in an error state. (#6510, #6485) (thanks @pawarren!)
• Slow local startup no longer makes a healthy server look unreachable. (#6475)
• A mutator that throws while being replayed no longer disconnects the client in a loop. The mutation is still sent to the server. (#6555)
• Client memory no longer grows with every query used in a session. (#6474)
• useSuspenseQuery with an options object and no suspendUntil now suspends until the first rows arrive, as documented (see Breaking Changes). (ffe67b23a) (thanks @charpeni!)
• Routine reconnects, auth errors and server rebalancing are logged as warnings or info instead of errors, and fatal errors now include their cause. (#6581, #6692, #6498, #6682, #6691, 42316d521, 075b4215e, 79adc09fb) (thanks @pawarren!)

zero-cache Reliability

• Fixed memory and resource leaks in zero-cache around closed connections, stopped view-syncers, abandoned hydrations, and inspector and profiling requests. (#6500, #6514, #6515, #6516, #6525, #6526, #6527, #6528, #6540, #6541, #6542, #6698)
• Fixed view-syncer crashes and Internal errors when another view-syncer advanced the same client group, when a client connection was missing, or when a push arrived on a closing connection. (#6481, #6530, #6600, #6614, #6631, d545bc7e2) (thanks @jdnvn!)
• Servers using zeroNodePg no longer crash when Postgres closes a pooled connection. (1b220fa07) (thanks @pawarren!)
• Analyzing a query in the Inspector no longer runs a view-syncer out of memory on large tables. (#6596)
• Error messages no longer include row values, query literals or mutation arguments. (#6413)

Replication, Backfills, and Backups

• Replication no longer rolls back or stops after an interrupted stream, and a replication-manager whose slot was invalidated now resyncs instead of crash-looping. (#6562, #6590)
• Backfills no longer apply rows before replication has caught up to them, resume when a view-syncer moves to another replication-manager, and no longer leak a Postgres connection that holds back vacuum. (#6685, #6684, #6513)
• Replication slots are more robust: creation retries while long transactions block it, slots stay active during long initial syncs, slots dropped outside Zero are handled, and inactive slots are cleaned up. (25a9ea1ac, f3bba7d73, 7a998b566, caf79b161, #6568)
• Replication lag is now measured correctly on Aurora. (e6a2590db)
• After initial sync, the replication-manager waits for a backup that covers its replica before serving, so restoring view-syncers no longer fall back to Postgres. (#6465)
• Fewer unnecessary query pipeline resets, and dropping an upstream index now refreshes table schemas. (4341cdcbd, 89787f827, bb71cc931)
• View-syncers fail fast when the replication-manager is unreachable, and a slow handshake is no longer mistaken for a missed heartbeat. (0af1a7109, #6598)
• During a resync, the old replication-manager keeps serving until the new one takes over. (21f349135)
• Change-database connections no longer leak after a failed catch-up, and a change-database reset no longer hangs. (#6529, #6632)
• Litestream's upload concurrency now scales with available memory, avoiding out-of-memory crashes in small containers. (240c05730)
• Configuration values that start with a dash, such as some shard IDs, are now parsed correctly. (81e6ac570)
• In the zero Docker image, initial sync and backfills no longer leave a stalled Postgres connection open after copying a table. Other installs keep the 1.10 behavior. (3f139f52a, 22fa3c27e)

Type Checking

• A rejected whereExists(..., {scalar: true}) now explains why, and a misspelled relationship name lists the valid ones. (cc020b368, 12a36dcfe)

Breaking Changes

• Legacy queries: zero-cache no longer accepts queries sent directly by clients. If you still use legacy queries, set ZERO_ALLOW_LEGACY_QUERIES=true or move to custom queries. (#6399)
• New states: code that switches over every result type or connection state must handle 'cached' and 'initializing'. Treat 'cached' like 'unknown': the data may be out of date. (#6578, #6561)
• Litestream v5 by default: remove the ZERO_LITESTREAM_VFS_* options, which no longer exist. Each replication-manager backs up to its own folder under the backup URL, so end the URL with / and add a lifecycle policy to your bucket. To keep v4 backups, set ZERO_LITESTREAM_BACKUP_USING_V5=false. If you set ZERO_LITESTREAM_EXECUTABLE_V5, use a Litestream build with -fork-to-url. (fb536f686, c1a586810)
• Changed defaults: inactive client groups are kept for 168 hours instead of 48, replication lag is reported every 10 seconds instead of 30, and ZERO_CHANGE_STREAMER_FLOW_CONTROL_CONSENSUS_TIMEOUT_PROPORTION is 4 instead of 2. Startup can take longer while zero-cache waits for a backup and for replication to catch up; give readiness probes more time. (#6484, 1c5ddcdb4, #6521, #6465)
• Server mutators: mutator return values must be JSON. A custom Database must pass addRetryPredicate on to the transaction it creates for tx.retryOn() to work. Using tx after its transaction ended now throws. (7756eb9bb, 4fe91c0d8)
• ZQLDatabase.run() no longer wraps reads in a transaction, so per-transaction settings such as SET LOCAL no longer apply to it. (#6577)
• useSuspenseQuery with an options object and no suspendUntil now suspends until the first rows arrive, as documented. Pass suspendUntil: 'complete' for the old behavior. (ffe67b23a)
• Upgrading from 1.9 or earlier: clients start with a new local database, and mutations they had not yet sent are lost. Clients upgrading from 1.10 keep theirs. (d597c69e8, 28e581483)
• Client storage: a client whose local store is corrupt now deletes it and calls onClientStateNotFound. Mutations that were not yet sent are lost. (#6558)
• One-time query reload: queries are identified differently in 1.11, so after an upgrade, and again after a rollback, zero-cache transforms and hydrates each client's queries once more. Expect a burst of requests to ZERO_QUERY_URL. (#6450, #6462)
• Deprecated: ZERO_CHANGE_STREAMER_MODE; use ZERO_CHANGE_STREAMER_URI. (698286b55)
• Events: a new zero/events/status/view-syncer/v1 event type. Event consumers that reject unknown types must accept it. (#6621)