From ed1ebc99b8c69ee4073118a37a44f6cbe743fcd8 Mon Sep 17 00:00:00 2001 From: joshuabrink Date: Wed, 8 Apr 2026 11:09:09 +0200 Subject: [PATCH 1/4] Improve upload data behavior details + cross-reference sdk pages --- client-sdks/reference/capacitor.mdx | 2 +- client-sdks/reference/dotnet.mdx | 2 +- client-sdks/reference/flutter.mdx | 2 +- client-sdks/reference/javascript-web.mdx | 2 +- client-sdks/reference/kotlin.mdx | 2 +- client-sdks/reference/node.mdx | 2 +- .../reference/react-native-and-expo.mdx | 3 +- client-sdks/reference/rust.mdx | 2 +- client-sdks/reference/swift.mdx | 2 +- client-sdks/reference/tauri.mdx | 2 +- client-sdks/writing-data.mdx | 4 +- .../app-backend/client-side-integration.mdx | 84 ++++++++++++++++++- handling-writes/writing-client-changes.mdx | 2 +- 13 files changed, 97 insertions(+), 14 deletions(-) diff --git a/client-sdks/reference/capacitor.mdx b/client-sdks/reference/capacitor.mdx index 354570dc..6fac7e67 100644 --- a/client-sdks/reference/capacitor.mdx +++ b/client-sdks/reference/capacitor.mdx @@ -186,7 +186,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/dotnet.mdx b/client-sdks/reference/dotnet.mdx index 35462793..aa5cf8a8 100644 --- a/client-sdks/reference/dotnet.mdx +++ b/client-sdks/reference/dotnet.mdx @@ -205,7 +205,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.FetchCredentials](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/NodeConnector.cs#L50) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.UploadData](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/NodeConnector.cs#L72) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.UploadData](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/NodeConnector.cs#L72) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/flutter.mdx b/client-sdks/reference/flutter.mdx index 61b2096c..b9655118 100644 --- a/client-sdks/reference/flutter.mdx +++ b/client-sdks/reference/flutter.mdx @@ -189,7 +189,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/fetchCredentials.html) \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/uploadData.html) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/uploadData.html) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/javascript-web.mdx b/client-sdks/reference/javascript-web.mdx index 7507d50d..7acace06 100644 --- a/client-sdks/reference/javascript-web.mdx +++ b/client-sdks/reference/javascript-web.mdx @@ -191,7 +191,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/kotlin.mdx b/client-sdks/reference/kotlin.mdx index 47bebcca..b88515de 100644 --- a/client-sdks/reference/kotlin.mdx +++ b/client-sdks/reference/kotlin.mdx @@ -164,7 +164,7 @@ Create a connector to integrate with your backend. The PowerSync backend connect Accordingly, the connector must implement two methods: 1. `PowerSyncBackendConnector.fetchCredentials` \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `PowerSyncBackendConnector.uploadData` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `PowerSyncBackendConnector.uploadData` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/node.mdx b/client-sdks/reference/node.mdx index 3d198be6..a8f76478 100644 --- a/client-sdks/reference/node.mdx +++ b/client-sdks/reference/node.mdx @@ -101,7 +101,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/react-native-and-expo.mdx b/client-sdks/reference/react-native-and-expo.mdx index 9c24e331..387010e1 100644 --- a/client-sdks/reference/react-native-and-expo.mdx +++ b/client-sdks/reference/react-native-and-expo.mdx @@ -186,8 +186,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) \- Use this to upload client-side changes to your app backend. -\-> See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) \- Use this to upload client-side changes to your app backend. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/rust.mdx b/client-sdks/reference/rust.mdx index 0f7caffe..da4ba98d 100644 --- a/client-sdks/reference/rust.mdx +++ b/client-sdks/reference/rust.mdx @@ -254,7 +254,7 @@ Create a connector to integrate with your backend. The PowerSync backend connect Accordingly, the connector must implement two methods: 1. `fetch_credentials` \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/swift.mdx b/client-sdks/reference/swift.mdx index ddbf8afd..2a21e2f1 100644 --- a/client-sdks/reference/swift.mdx +++ b/client-sdks/reference/swift.mdx @@ -127,7 +127,7 @@ Accordingly, the connector must implement two methods: 1. `PowerSyncBackendConnectorProtocol.fetchCredentials` - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `PowerSyncBackendConnectorProtocol.uploadData` - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `PowerSyncBackendConnectorProtocol.uploadData` - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/tauri.mdx b/client-sdks/reference/tauri.mdx index 109e4993..9dd9f90e 100644 --- a/client-sdks/reference/tauri.mdx +++ b/client-sdks/reference/tauri.mdx @@ -189,7 +189,7 @@ Please [let us know](/resources/contact-us) if you want to implement a backend c Accordingly, the connector must implement two methods: 1. `fetch_credentials` \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/writing-data.mdx b/client-sdks/writing-data.mdx index d8da087a..14efc0c5 100644 --- a/client-sdks/writing-data.mdx +++ b/client-sdks/writing-data.mdx @@ -126,7 +126,9 @@ PowerSync automatically queues writes and uploads them to your backend. The uplo | `PATCH` | Update existing row | Contains the row `id`, and value of each changed column. | Generated by `UPDATE` statements. | | `DELETE` | Delete existing row | Contains the row `id` | Generated by `DELETE` statements. | -For details on how writes are uploaded to your backend, see [Writing Client Changes](/handling-writes/writing-client-changes). +The SDK automatically processes this queue by calling your `uploadData()` function. Uploads are triggered after local writes, on connection/reconnection, and on periodic keepalive messages. For a detailed breakdown of when and how `uploadData()` is called, including throttling, retry behavior, and error handling, see [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called). + +For details on how to structure your backend to accept these writes, see [Writing Client Changes](/handling-writes/writing-client-changes). ## Advanced Topics diff --git a/configuration/app-backend/client-side-integration.mdx b/configuration/app-backend/client-side-integration.mdx index 70ddb09f..cff6b5e2 100644 --- a/configuration/app-backend/client-side-integration.mdx +++ b/configuration/app-backend/client-side-integration.mdx @@ -20,11 +20,93 @@ Accordingly, you must pass a _backend connector_ as an argument when you call `c | Purpose | Function | Description | |---------|----------|-------------| -| **Uploading mutations to your backend:** | `uploadData()` | The PowerSync Client SDK automatically calls this function to upload client-side mutations to your backend. Whenever you write to the client-side SQLite database, those writes are also automatically placed into an _upload queue_ by the Client SDK, and the Client SDK processes the entries in the upload queue by calling `uploadData()`. You should define your `uploadData()` function to call your backend application API to upload and apply the write operations to your backend source database. The Client SDK automatically handles retries in the case of failures. See [Writing Data](/client-sdks/writing-data) in the _Client SDKs_ section for more details on the implementation of `uploadData()`. | +| **Uploading mutations to your backend:** | `uploadData()` | The PowerSync Client SDK automatically calls this function to upload client-side mutations to your backend. Whenever you write to the client-side SQLite database, those writes are also automatically placed into an _upload queue_ by the Client SDK, and the Client SDK processes the entries in the upload queue by calling `uploadData()`. You should define your `uploadData()` function to call your backend application API to upload and apply the write operations to your backend source database. The Client SDK automatically handles retries in the case of failures. See the [detailed behavior below](#when-uploaddata-is-called) and [Writing Data](/client-sdks/writing-data) in the _Client SDKs_ section for more details on the implementation of `uploadData()`. | | **Authentication integration:** | `fetchCredentials()` | Called by the PowerSync Client SDK to obtain a JWT and the endpoint URL for your PowerSync Service instance. The SDK uses the JWT to authenticate against the PowerSync Service. `fetchCredentials()` typically returns an object with `token` (JWT) and `endpoint` fields. See [Authentication Setup](/configuration/auth/overview) for more details on JWT authentication. | Some authentication providers generate JWTs for users which PowerSync can verify directly, and in that case, your `fetchCredentials()` function implementation can simply return that JWT from client-side state. Your `fetchCredentials()` implementation only needs to retrieve a JWT from your backend if you are using [Custom Authentication](/configuration/auth/custom) integration. See the [Authentication Overview](/configuration/auth/overview) for more background. +### When `uploadData()` is Called + +The PowerSync Client SDK calls `uploadData()` automatically - you never call it directly. It is invoked in the following scenarios: + +1. **After a local write** - Any `INSERT`, `UPDATE`, or `DELETE` on a PowerSync table adds an entry to the internal upload queue (the `ps_crud` table). The SDK monitors this table for changes and triggers `uploadData()` shortly after. +2. **On initial connection / reconnection** - When `connect()` establishes (or re-establishes) a sync stream and the first message is received from the PowerSync Service, the SDK triggers `uploadData()` to flush any writes that were made while offline or disconnected. +3. **On keepalive messages** - The PowerSync Service sends periodic keepalive messages (roughly every 20 seconds). Each keepalive triggers an upload attempt, ensuring pending writes are retried even if no new local writes have occurred. +4. **After an error, with retry** - If `uploadData()` throws an error, the SDK waits for a configurable delay (default: 5 seconds, controlled by `retryDelayMs`) and then retries. This continues until the upload succeeds or the sync stream is disconnected. + +#### Upload Loop Behavior + +The SDK calls `uploadData()` in a **loop** - not just once per trigger. After each successful `uploadData()` call, the SDK checks whether there are more items in the upload queue. If there are, it calls `uploadData()` again immediately. The loop continues until the queue is empty. This means your `uploadData()` implementation only needs to process one batch (or one transaction) per call. + +Once the queue is empty, the SDK updates an internal write checkpoint used for [consistency](/architecture/consistency) tracking. + +#### Throttling + +To avoid excessive calls, upload triggers are **throttled**. Rapid local writes (e.g., multiple `INSERT` statements in quick succession) are coalesced so that `uploadData()` is invoked at most once per throttle interval (default: 1 second). If an upload is already in progress when a new write occurs, the SDK will trigger another upload after the current one completes. + +The throttle interval is configurable via options when calling `connect()`: + + +```typescript TypeScript +await db.connect(connector, { crudUploadThrottleMs: 500 }); +``` + +```dart Dart/Flutter +db.connect( + connector: connector, + options: SyncOptions( + crudThrottleTime: Duration(milliseconds: 500), + ), +); +``` + +```kotlin Kotlin +database.connect(connector, crudThrottleMs = 500L) +``` + +```swift Swift +try await database.connect( + connector: connector, + options: ConnectOptions(crudThrottle: 0.5) +) +``` + +```csharp .NET +await database.Connect(connector, new PowerSyncConnectionOptions( + crudUploadThrottleMs: 500 +)); +``` + +```rust Rust +// Rust SDK does not currently expose a CRUD throttle option +db.connect(SyncOptions::new(connector)).await; +``` + + +#### Error Handling + + +Your backend API must return a `2xx` response for validation errors or write conflicts. A `4xx` or `5xx` error response from your backend will cause the SDK to **retry** the same upload indefinitely, effectively blocking the upload queue. See [Handling Write / Validation Errors](/handling-writes/handling-write-validation-errors) for recommended patterns. + + +When `uploadData()` throws an error: +1. The SDK logs the error and updates the sync status with an `uploadError`. +2. It waits for the retry delay (default: 5 seconds). +3. It retries the upload from the beginning of the queue. +4. If the sync stream disconnects during the retry wait, the upload loop exits and will resume when the connection is re-established. + +#### Stalled Upload Queue Detection + +If the SDK detects that the same CRUD entry is at the front of the queue across consecutive upload iterations (i.e., `uploadData()` returned without error but didn't call `.complete()` on the batch or transaction), it logs a warning: + +> _"Potentially previously uploaded CRUD entries are still present in the upload queue. Make sure to handle uploads and complete CRUD transactions or batches by calling and awaiting their `.complete()` method."_ + +This typically means your `uploadData()` implementation is not calling `.complete()` after successfully processing entries. Always call `.complete()` on the `CrudBatch` or `CrudTransaction` to remove processed entries from the queue. + + +`uploadData()` is only called while the sync stream is connected. If the device is offline, writes accumulate in the upload queue and are uploaded automatically when connectivity is restored and the sync stream reconnects. + + ### When `fetchCredentials()` is Called The PowerSync Client SDK **caches credentials internally** and `fetchCredentials` is called in the following scenarios: diff --git a/handling-writes/writing-client-changes.mdx b/handling-writes/writing-client-changes.mdx index df6a4457..28f7565a 100644 --- a/handling-writes/writing-client-changes.mdx +++ b/handling-writes/writing-client-changes.mdx @@ -4,7 +4,7 @@ description: "Build a backend API endpoint to apply client-side writes from the --- - Your backend application receives the write operations based on how you defined your `uploadData()` function in the `PowerSyncBackendConnector` in your client-side app. See [Client-Side Integration](/configuration/app-backend/client-side-integration) for details. + Your backend application receives the write operations based on how you defined your `uploadData()` function in the `PowerSyncBackendConnector` in your client-side app. See [Client-Side Integration](/configuration/app-backend/client-side-integration) for details, including [when `uploadData()` is called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) and its retry behavior. From 7e02cd3ed8eba3770b235896f9cfbd41f0f5fbea Mon Sep 17 00:00:00 2001 From: joshuabrink Date: Wed, 8 Apr 2026 11:50:58 +0200 Subject: [PATCH 2/4] Correct throttle defaults, stalled queue behavior, and cross-references --- architecture/client-architecture.mdx | 2 +- .../reference/react-native-and-expo.mdx | 2 +- client-sdks/writing-data.mdx | 2 +- .../app-backend/client-side-integration.mdx | 27 ++++++++++++------- 4 files changed, 20 insertions(+), 13 deletions(-) diff --git a/architecture/client-architecture.mdx b/architecture/client-architecture.mdx index a2a3bd35..2fe13ef3 100644 --- a/architecture/client-architecture.mdx +++ b/architecture/client-architecture.mdx @@ -136,7 +136,7 @@ The Client SDK processes the upload queue by invoking an `uploadData()` function The reason why we designed PowerSync this way is that it allows you to apply your own backend business logic, validations and authorization to any mutations going to your source database. -The PowerSync Client SDK automatically takes care of network failures and retries. If processing mutations in the upload queue fails (e.g. because the user is offline), it is automatically retried. +The PowerSync Client SDK automatically takes care of network failures and retries. If processing mutations in the upload queue fails (e.g. because the user is offline), it is automatically retried. For a detailed breakdown of when `uploadData()` is called, including throttling, retry behavior, and error handling, see [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called). diff --git a/client-sdks/reference/react-native-and-expo.mdx b/client-sdks/reference/react-native-and-expo.mdx index 387010e1..852dd3f8 100644 --- a/client-sdks/reference/react-native-and-expo.mdx +++ b/client-sdks/reference/react-native-and-expo.mdx @@ -186,7 +186,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) \- Use this to upload client-side changes to your app backend. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/writing-data.mdx b/client-sdks/writing-data.mdx index 14efc0c5..586c5e23 100644 --- a/client-sdks/writing-data.mdx +++ b/client-sdks/writing-data.mdx @@ -126,7 +126,7 @@ PowerSync automatically queues writes and uploads them to your backend. The uplo | `PATCH` | Update existing row | Contains the row `id`, and value of each changed column. | Generated by `UPDATE` statements. | | `DELETE` | Delete existing row | Contains the row `id` | Generated by `DELETE` statements. | -The SDK automatically processes this queue by calling your `uploadData()` function. Uploads are triggered after local writes, on connection/reconnection, and on periodic keepalive messages. For a detailed breakdown of when and how `uploadData()` is called, including throttling, retry behavior, and error handling, see [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called). +The SDK automatically processes this queue by calling your `uploadData()` function. For a detailed breakdown of when and how `uploadData()` is called, including triggers, throttling, retry behavior, and error handling, see [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called). For details on how to structure your backend to accept these writes, see [Writing Client Changes](/handling-writes/writing-client-changes). diff --git a/configuration/app-backend/client-side-integration.mdx b/configuration/app-backend/client-side-integration.mdx index cff6b5e2..0576503f 100644 --- a/configuration/app-backend/client-side-integration.mdx +++ b/configuration/app-backend/client-side-integration.mdx @@ -31,7 +31,7 @@ The PowerSync Client SDK calls `uploadData()` automatically - you never call it 1. **After a local write** - Any `INSERT`, `UPDATE`, or `DELETE` on a PowerSync table adds an entry to the internal upload queue (the `ps_crud` table). The SDK monitors this table for changes and triggers `uploadData()` shortly after. 2. **On initial connection / reconnection** - When `connect()` establishes (or re-establishes) a sync stream and the first message is received from the PowerSync Service, the SDK triggers `uploadData()` to flush any writes that were made while offline or disconnected. -3. **On keepalive messages** - The PowerSync Service sends periodic keepalive messages (roughly every 20 seconds). Each keepalive triggers an upload attempt, ensuring pending writes are retried even if no new local writes have occurred. +3. **On keepalive messages** - The PowerSync Service sends periodic keepalive messages (every 20 seconds, with slight jitter). Each keepalive triggers an upload attempt, ensuring pending writes are retried even if no new local writes have occurred. 4. **After an error, with retry** - If `uploadData()` throws an error, the SDK waits for a configurable delay (default: 5 seconds, controlled by `retryDelayMs`) and then retries. This continues until the upload succeeds or the sync stream is disconnected. #### Upload Loop Behavior @@ -42,7 +42,17 @@ Once the queue is empty, the SDK updates an internal write checkpoint used for [ #### Throttling -To avoid excessive calls, upload triggers are **throttled**. Rapid local writes (e.g., multiple `INSERT` statements in quick succession) are coalesced so that `uploadData()` is invoked at most once per throttle interval (default: 1 second). If an upload is already in progress when a new write occurs, the SDK will trigger another upload after the current one completes. +To avoid excessive calls, upload triggers are **throttled**. Rapid local writes (e.g., multiple `INSERT` statements in quick succession) are coalesced so that `uploadData()` is invoked at most once per throttle interval. If an upload is already in progress when a new write occurs, the SDK will trigger another upload after the current one completes. + +The default throttle interval varies by SDK: + +| SDK | Default | Option Name | +|-----|---------|-------------| +| JavaScript / React Native / Node / Capacitor / Tauri | 1,000 ms | `crudUploadThrottleMs` | +| Kotlin | 1,000 ms | `crudThrottleMs` | +| Swift | 1 second | `crudThrottle` (seconds) | +| .NET | 1,000 ms | `CrudUploadThrottleMs` | +| Dart / Flutter | 10 ms | `crudThrottleTime` | The throttle interval is configurable via options when calling `connect()`: @@ -67,7 +77,7 @@ database.connect(connector, crudThrottleMs = 500L) ```swift Swift try await database.connect( connector: connector, - options: ConnectOptions(crudThrottle: 0.5) + options: ConnectOptions(crudThrottle: 0.5) // seconds ) ``` @@ -76,17 +86,12 @@ await database.Connect(connector, new PowerSyncConnectionOptions( crudUploadThrottleMs: 500 )); ``` - -```rust Rust -// Rust SDK does not currently expose a CRUD throttle option -db.connect(SyncOptions::new(connector)).await; -``` #### Error Handling -Your backend API must return a `2xx` response for validation errors or write conflicts. A `4xx` or `5xx` error response from your backend will cause the SDK to **retry** the same upload indefinitely, effectively blocking the upload queue. See [Handling Write / Validation Errors](/handling-writes/handling-write-validation-errors) for recommended patterns. +If your `uploadData()` throws an error (e.g. due to a `4xx` or `5xx` response from your backend), the SDK will **retry** the same upload indefinitely, effectively blocking the upload queue. Your backend should return `2xx` for validation errors or write conflicts, and reserve error responses for transient failures. See [Writing Client Changes](/handling-writes/writing-client-changes#recommendations) and [Handling Write / Validation Errors](/handling-writes/handling-write-validation-errors) for recommended patterns. When `uploadData()` throws an error: @@ -99,7 +104,9 @@ When `uploadData()` throws an error: If the SDK detects that the same CRUD entry is at the front of the queue across consecutive upload iterations (i.e., `uploadData()` returned without error but didn't call `.complete()` on the batch or transaction), it logs a warning: -> _"Potentially previously uploaded CRUD entries are still present in the upload queue. Make sure to handle uploads and complete CRUD transactions or batches by calling and awaiting their `.complete()` method."_ +> _"Potentially previously uploaded CRUD entries are still present in the upload queue. Make sure to handle uploads and complete CRUD transactions or batches by calling and awaiting their \[.complete()\] method. The next upload iteration will be delayed."_ + +After logging this warning, the SDK **throws an internal error**, which causes it to enter the retry delay path (default: 5 seconds) before attempting the upload again. This prevents a tight loop when `uploadData()` consistently fails to process entries. This typically means your `uploadData()` implementation is not calling `.complete()` after successfully processing entries. Always call `.complete()` on the `CrudBatch` or `CrudTransaction` to remove processed entries from the queue. From 2e04aec265de4885930e3fb87a443c8900193773 Mon Sep 17 00:00:00 2001 From: joshuabrink Date: Thu, 9 Apr 2026 12:13:44 +0200 Subject: [PATCH 3/4] Improve wording --- client-sdks/reference/capacitor.mdx | 2 +- client-sdks/reference/dotnet.mdx | 2 +- client-sdks/reference/flutter.mdx | 2 +- client-sdks/reference/javascript-web.mdx | 2 +- client-sdks/reference/kotlin.mdx | 2 +- client-sdks/reference/node.mdx | 2 +- client-sdks/reference/react-native-and-expo.mdx | 2 +- client-sdks/reference/rust.mdx | 2 +- client-sdks/reference/swift.mdx | 2 +- client-sdks/reference/tauri.mdx | 2 +- 10 files changed, 10 insertions(+), 10 deletions(-) diff --git a/client-sdks/reference/capacitor.mdx b/client-sdks/reference/capacitor.mdx index 6fac7e67..68ced63c 100644 --- a/client-sdks/reference/capacitor.mdx +++ b/client-sdks/reference/capacitor.mdx @@ -186,7 +186,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/dotnet.mdx b/client-sdks/reference/dotnet.mdx index aa5cf8a8..9b181d3a 100644 --- a/client-sdks/reference/dotnet.mdx +++ b/client-sdks/reference/dotnet.mdx @@ -205,7 +205,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.FetchCredentials](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/NodeConnector.cs#L50) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.UploadData](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/NodeConnector.cs#L72) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.UploadData](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/NodeConnector.cs#L72) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/flutter.mdx b/client-sdks/reference/flutter.mdx index b9655118..cb78dd7c 100644 --- a/client-sdks/reference/flutter.mdx +++ b/client-sdks/reference/flutter.mdx @@ -189,7 +189,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/fetchCredentials.html) \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/uploadData.html) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/uploadData.html) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/javascript-web.mdx b/client-sdks/reference/javascript-web.mdx index 7acace06..fdaf44f7 100644 --- a/client-sdks/reference/javascript-web.mdx +++ b/client-sdks/reference/javascript-web.mdx @@ -191,7 +191,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/kotlin.mdx b/client-sdks/reference/kotlin.mdx index b88515de..c140c8f4 100644 --- a/client-sdks/reference/kotlin.mdx +++ b/client-sdks/reference/kotlin.mdx @@ -164,7 +164,7 @@ Create a connector to integrate with your backend. The PowerSync backend connect Accordingly, the connector must implement two methods: 1. `PowerSyncBackendConnector.fetchCredentials` \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `PowerSyncBackendConnector.uploadData` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `PowerSyncBackendConnector.uploadData` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/node.mdx b/client-sdks/reference/node.mdx index a8f76478..6660c5f3 100644 --- a/client-sdks/reference/node.mdx +++ b/client-sdks/reference/node.mdx @@ -101,7 +101,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/react-native-and-expo.mdx b/client-sdks/reference/react-native-and-expo.mdx index 852dd3f8..3806b2c9 100644 --- a/client-sdks/reference/react-native-and-expo.mdx +++ b/client-sdks/reference/react-native-and-expo.mdx @@ -186,7 +186,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/rust.mdx b/client-sdks/reference/rust.mdx index da4ba98d..2f62c863 100644 --- a/client-sdks/reference/rust.mdx +++ b/client-sdks/reference/rust.mdx @@ -254,7 +254,7 @@ Create a connector to integrate with your backend. The PowerSync backend connect Accordingly, the connector must implement two methods: 1. `fetch_credentials` \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/swift.mdx b/client-sdks/reference/swift.mdx index 2a21e2f1..79697854 100644 --- a/client-sdks/reference/swift.mdx +++ b/client-sdks/reference/swift.mdx @@ -127,7 +127,7 @@ Accordingly, the connector must implement two methods: 1. `PowerSyncBackendConnectorProtocol.fetchCredentials` - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `PowerSyncBackendConnectorProtocol.uploadData` - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `PowerSyncBackendConnectorProtocol.uploadData` - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/tauri.mdx b/client-sdks/reference/tauri.mdx index 9dd9f90e..d7fd7763 100644 --- a/client-sdks/reference/tauri.mdx +++ b/client-sdks/reference/tauri.mdx @@ -189,7 +189,7 @@ Please [let us know](/resources/contact-us) if you want to implement a backend c Accordingly, the connector must implement two methods: 1. `fetch_credentials` \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: From b19416009c34d5841f3ef854d0766785e771c4de Mon Sep 17 00:00:00 2001 From: joshuabrink Date: Wed, 29 Apr 2026 14:34:45 +0200 Subject: [PATCH 4/4] Expand on upload data detection --- .../app-backend/client-side-integration.mdx | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/configuration/app-backend/client-side-integration.mdx b/configuration/app-backend/client-side-integration.mdx index 0576503f..5d85b647 100644 --- a/configuration/app-backend/client-side-integration.mdx +++ b/configuration/app-backend/client-side-integration.mdx @@ -29,11 +29,24 @@ Accordingly, you must pass a _backend connector_ as an argument when you call `c The PowerSync Client SDK calls `uploadData()` automatically - you never call it directly. It is invoked in the following scenarios: -1. **After a local write** - Any `INSERT`, `UPDATE`, or `DELETE` on a PowerSync table adds an entry to the internal upload queue (the `ps_crud` table). The SDK monitors this table for changes and triggers `uploadData()` shortly after. +1. **After a local write** - Any `INSERT`, `UPDATE`, or `DELETE` on a PowerSync table adds an entry to the internal upload queue (the `ps_crud` table). The SDK monitors this table for changes and triggers `uploadData()` shortly after. See [How Local Writes Are Detected](#how-local-writes-are-detected) below for details on the underlying mechanism. 2. **On initial connection / reconnection** - When `connect()` establishes (or re-establishes) a sync stream and the first message is received from the PowerSync Service, the SDK triggers `uploadData()` to flush any writes that were made while offline or disconnected. 3. **On keepalive messages** - The PowerSync Service sends periodic keepalive messages (every 20 seconds, with slight jitter). Each keepalive triggers an upload attempt, ensuring pending writes are retried even if no new local writes have occurred. 4. **After an error, with retry** - If `uploadData()` throws an error, the SDK waits for a configurable delay (default: 5 seconds, controlled by `retryDelayMs`) and then retries. This continues until the upload succeeds or the sync stream is disconnected. +#### How Local Writes Are Detected + +By default, each PowerSync-managed table is exposed in SQLite as a **view** with `INSTEAD OF INSERT/UPDATE/DELETE` triggers (generated by the PowerSync SQLite extension when your [client-side schema](/intro/setup-guide#define-your-client-side-schema) is applied). When you write to a PowerSync table, those triggers do two things atomically: + +1. Apply the change to the [underlying `ps_data__` table](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure). +2. Append an entry to `ps_crud` describing the operation (see [Write Operations and Upload Queue](/client-sdks/writing-data#write-operations-and-upload-queue) for the entry format). + +Because both happen in the same transaction, the upload queue can never get out of sync with local data. ([Raw Tables](/client-sdks/advanced/raw-tables#capture-local-writes-with-triggers) preserve this guarantee, except you create the triggers yourself, typically via the `powersync_create_raw_table_crud_trigger` helper.) + +The SDK then detects new `ps_crud` entries by subscribing to SQLite's table-update notifications, filtered for changes to `ps_crud`. This is the same mechanism that powers reactive [watch queries](/client-sdks/watch-queries). When a change is observed, the SDK schedules an `uploadData()` call subject to the configured [throttle interval](#throttling). + +The mechanism is consistent across all PowerSync SDKs (JavaScript, Dart, Kotlin, Swift, .NET, Rust), since the triggers and `ps_crud` table are defined by the shared [PowerSync SQLite core extension](https://github.com/powersync-ja/powersync-sqlite-core). + #### Upload Loop Behavior The SDK calls `uploadData()` in a **loop** - not just once per trigger. After each successful `uploadData()` call, the SDK checks whether there are more items in the upload queue. If there are, it calls `uploadData()` again immediately. The loop continues until the queue is empty. This means your `uploadData()` implementation only needs to process one batch (or one transaction) per call.