diff --git a/architecture/client-architecture.mdx b/architecture/client-architecture.mdx index a2a3bd35..2fe13ef3 100644 --- a/architecture/client-architecture.mdx +++ b/architecture/client-architecture.mdx @@ -136,7 +136,7 @@ The Client SDK processes the upload queue by invoking an `uploadData()` function The reason why we designed PowerSync this way is that it allows you to apply your own backend business logic, validations and authorization to any mutations going to your source database. -The PowerSync Client SDK automatically takes care of network failures and retries. If processing mutations in the upload queue fails (e.g. because the user is offline), it is automatically retried. +The PowerSync Client SDK automatically takes care of network failures and retries. If processing mutations in the upload queue fails (e.g. because the user is offline), it is automatically retried. For a detailed breakdown of when `uploadData()` is called, including throttling, retry behavior, and error handling, see [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called). diff --git a/client-sdks/reference/capacitor.mdx b/client-sdks/reference/capacitor.mdx index 354570dc..68ced63c 100644 --- a/client-sdks/reference/capacitor.mdx +++ b/client-sdks/reference/capacitor.mdx @@ -186,7 +186,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/dotnet.mdx b/client-sdks/reference/dotnet.mdx index 35462793..9b181d3a 100644 --- a/client-sdks/reference/dotnet.mdx +++ b/client-sdks/reference/dotnet.mdx @@ -205,7 +205,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.FetchCredentials](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/NodeConnector.cs#L50) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.UploadData](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/NodeConnector.cs#L72) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.UploadData](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/NodeConnector.cs#L72) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/flutter.mdx b/client-sdks/reference/flutter.mdx index 61b2096c..cb78dd7c 100644 --- a/client-sdks/reference/flutter.mdx +++ b/client-sdks/reference/flutter.mdx @@ -189,7 +189,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/fetchCredentials.html) \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/uploadData.html) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/uploadData.html) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/javascript-web.mdx b/client-sdks/reference/javascript-web.mdx index 7507d50d..fdaf44f7 100644 --- a/client-sdks/reference/javascript-web.mdx +++ b/client-sdks/reference/javascript-web.mdx @@ -191,7 +191,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/kotlin.mdx b/client-sdks/reference/kotlin.mdx index 47bebcca..c140c8f4 100644 --- a/client-sdks/reference/kotlin.mdx +++ b/client-sdks/reference/kotlin.mdx @@ -164,7 +164,7 @@ Create a connector to integrate with your backend. The PowerSync backend connect Accordingly, the connector must implement two methods: 1. `PowerSyncBackendConnector.fetchCredentials` \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `PowerSyncBackendConnector.uploadData` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `PowerSyncBackendConnector.uploadData` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/node.mdx b/client-sdks/reference/node.mdx index 3d198be6..6660c5f3 100644 --- a/client-sdks/reference/node.mdx +++ b/client-sdks/reference/node.mdx @@ -101,7 +101,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/react-native-and-expo.mdx b/client-sdks/reference/react-native-and-expo.mdx index 9c24e331..3806b2c9 100644 --- a/client-sdks/reference/react-native-and-expo.mdx +++ b/client-sdks/reference/react-native-and-expo.mdx @@ -186,8 +186,7 @@ The PowerSync backend connector provides the connection between your application Accordingly, the connector must implement two methods: 1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) \- Use this to upload client-side changes to your app backend. -\-> See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/rust.mdx b/client-sdks/reference/rust.mdx index 0f7caffe..2f62c863 100644 --- a/client-sdks/reference/rust.mdx +++ b/client-sdks/reference/rust.mdx @@ -254,7 +254,7 @@ Create a connector to integrate with your backend. The PowerSync backend connect Accordingly, the connector must implement two methods: 1. `fetch_credentials` \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/swift.mdx b/client-sdks/reference/swift.mdx index ddbf8afd..79697854 100644 --- a/client-sdks/reference/swift.mdx +++ b/client-sdks/reference/swift.mdx @@ -127,7 +127,7 @@ Accordingly, the connector must implement two methods: 1. `PowerSyncBackendConnectorProtocol.fetchCredentials` - This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `PowerSyncBackendConnectorProtocol.uploadData` - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `PowerSyncBackendConnectorProtocol.uploadData` - This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/reference/tauri.mdx b/client-sdks/reference/tauri.mdx index 109e4993..d7fd7763 100644 --- a/client-sdks/reference/tauri.mdx +++ b/client-sdks/reference/tauri.mdx @@ -189,7 +189,7 @@ Please [let us know](/resources/contact-us) if you want to implement a backend c Accordingly, the connector must implement two methods: 1. `fetch_credentials` \- This method is automatically invoked by the PowerSync Client SDK to obtain authentication credentials. The SDK caches credentials internally and only calls this method when needed (e.g. on initial connection or when the token is near expiry). See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details, and [Authentication Setup](/configuration/auth/overview) for instructions on how the credentials should be generated. -2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app backend via your backend API. Therefore, in your implementation, you need to define how your backend API is called. See [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. +2. `upload_data` \- This method will be automatically invoked by the PowerSync Client SDK whenever it needs to upload client-side writes to your app's backend API. You need to implement how those writes are processed and uploaded in this method. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for details on triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for considerations on the app backend implementation. **Example**: diff --git a/client-sdks/writing-data.mdx b/client-sdks/writing-data.mdx index d8da087a..586c5e23 100644 --- a/client-sdks/writing-data.mdx +++ b/client-sdks/writing-data.mdx @@ -126,7 +126,9 @@ PowerSync automatically queues writes and uploads them to your backend. The uplo | `PATCH` | Update existing row | Contains the row `id`, and value of each changed column. | Generated by `UPDATE` statements. | | `DELETE` | Delete existing row | Contains the row `id` | Generated by `DELETE` statements. | -For details on how writes are uploaded to your backend, see [Writing Client Changes](/handling-writes/writing-client-changes). +The SDK automatically processes this queue by calling your `uploadData()` function. For a detailed breakdown of when and how `uploadData()` is called, including triggers, throttling, retry behavior, and error handling, see [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called). + +For details on how to structure your backend to accept these writes, see [Writing Client Changes](/handling-writes/writing-client-changes). ## Advanced Topics diff --git a/configuration/app-backend/client-side-integration.mdx b/configuration/app-backend/client-side-integration.mdx index 70ddb09f..5d85b647 100644 --- a/configuration/app-backend/client-side-integration.mdx +++ b/configuration/app-backend/client-side-integration.mdx @@ -20,11 +20,113 @@ Accordingly, you must pass a _backend connector_ as an argument when you call `c | Purpose | Function | Description | |---------|----------|-------------| -| **Uploading mutations to your backend:** | `uploadData()` | The PowerSync Client SDK automatically calls this function to upload client-side mutations to your backend. Whenever you write to the client-side SQLite database, those writes are also automatically placed into an _upload queue_ by the Client SDK, and the Client SDK processes the entries in the upload queue by calling `uploadData()`. You should define your `uploadData()` function to call your backend application API to upload and apply the write operations to your backend source database. The Client SDK automatically handles retries in the case of failures. See [Writing Data](/client-sdks/writing-data) in the _Client SDKs_ section for more details on the implementation of `uploadData()`. | +| **Uploading mutations to your backend:** | `uploadData()` | The PowerSync Client SDK automatically calls this function to upload client-side mutations to your backend. Whenever you write to the client-side SQLite database, those writes are also automatically placed into an _upload queue_ by the Client SDK, and the Client SDK processes the entries in the upload queue by calling `uploadData()`. You should define your `uploadData()` function to call your backend application API to upload and apply the write operations to your backend source database. The Client SDK automatically handles retries in the case of failures. See the [detailed behavior below](#when-uploaddata-is-called) and [Writing Data](/client-sdks/writing-data) in the _Client SDKs_ section for more details on the implementation of `uploadData()`. | | **Authentication integration:** | `fetchCredentials()` | Called by the PowerSync Client SDK to obtain a JWT and the endpoint URL for your PowerSync Service instance. The SDK uses the JWT to authenticate against the PowerSync Service. `fetchCredentials()` typically returns an object with `token` (JWT) and `endpoint` fields. See [Authentication Setup](/configuration/auth/overview) for more details on JWT authentication. | Some authentication providers generate JWTs for users which PowerSync can verify directly, and in that case, your `fetchCredentials()` function implementation can simply return that JWT from client-side state. Your `fetchCredentials()` implementation only needs to retrieve a JWT from your backend if you are using [Custom Authentication](/configuration/auth/custom) integration. See the [Authentication Overview](/configuration/auth/overview) for more background. +### When `uploadData()` is Called + +The PowerSync Client SDK calls `uploadData()` automatically - you never call it directly. It is invoked in the following scenarios: + +1. **After a local write** - Any `INSERT`, `UPDATE`, or `DELETE` on a PowerSync table adds an entry to the internal upload queue (the `ps_crud` table). The SDK monitors this table for changes and triggers `uploadData()` shortly after. See [How Local Writes Are Detected](#how-local-writes-are-detected) below for details on the underlying mechanism. +2. **On initial connection / reconnection** - When `connect()` establishes (or re-establishes) a sync stream and the first message is received from the PowerSync Service, the SDK triggers `uploadData()` to flush any writes that were made while offline or disconnected. +3. **On keepalive messages** - The PowerSync Service sends periodic keepalive messages (every 20 seconds, with slight jitter). Each keepalive triggers an upload attempt, ensuring pending writes are retried even if no new local writes have occurred. +4. **After an error, with retry** - If `uploadData()` throws an error, the SDK waits for a configurable delay (default: 5 seconds, controlled by `retryDelayMs`) and then retries. This continues until the upload succeeds or the sync stream is disconnected. + +#### How Local Writes Are Detected + +By default, each PowerSync-managed table is exposed in SQLite as a **view** with `INSTEAD OF INSERT/UPDATE/DELETE` triggers (generated by the PowerSync SQLite extension when your [client-side schema](/intro/setup-guide#define-your-client-side-schema) is applied). When you write to a PowerSync table, those triggers do two things atomically: + +1. Apply the change to the [underlying `ps_data__` table](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure). +2. Append an entry to `ps_crud` describing the operation (see [Write Operations and Upload Queue](/client-sdks/writing-data#write-operations-and-upload-queue) for the entry format). + +Because both happen in the same transaction, the upload queue can never get out of sync with local data. ([Raw Tables](/client-sdks/advanced/raw-tables#capture-local-writes-with-triggers) preserve this guarantee, except you create the triggers yourself, typically via the `powersync_create_raw_table_crud_trigger` helper.) + +The SDK then detects new `ps_crud` entries by subscribing to SQLite's table-update notifications, filtered for changes to `ps_crud`. This is the same mechanism that powers reactive [watch queries](/client-sdks/watch-queries). When a change is observed, the SDK schedules an `uploadData()` call subject to the configured [throttle interval](#throttling). + +The mechanism is consistent across all PowerSync SDKs (JavaScript, Dart, Kotlin, Swift, .NET, Rust), since the triggers and `ps_crud` table are defined by the shared [PowerSync SQLite core extension](https://github.com/powersync-ja/powersync-sqlite-core). + +#### Upload Loop Behavior + +The SDK calls `uploadData()` in a **loop** - not just once per trigger. After each successful `uploadData()` call, the SDK checks whether there are more items in the upload queue. If there are, it calls `uploadData()` again immediately. The loop continues until the queue is empty. This means your `uploadData()` implementation only needs to process one batch (or one transaction) per call. + +Once the queue is empty, the SDK updates an internal write checkpoint used for [consistency](/architecture/consistency) tracking. + +#### Throttling + +To avoid excessive calls, upload triggers are **throttled**. Rapid local writes (e.g., multiple `INSERT` statements in quick succession) are coalesced so that `uploadData()` is invoked at most once per throttle interval. If an upload is already in progress when a new write occurs, the SDK will trigger another upload after the current one completes. + +The default throttle interval varies by SDK: + +| SDK | Default | Option Name | +|-----|---------|-------------| +| JavaScript / React Native / Node / Capacitor / Tauri | 1,000 ms | `crudUploadThrottleMs` | +| Kotlin | 1,000 ms | `crudThrottleMs` | +| Swift | 1 second | `crudThrottle` (seconds) | +| .NET | 1,000 ms | `CrudUploadThrottleMs` | +| Dart / Flutter | 10 ms | `crudThrottleTime` | + +The throttle interval is configurable via options when calling `connect()`: + + +```typescript TypeScript +await db.connect(connector, { crudUploadThrottleMs: 500 }); +``` + +```dart Dart/Flutter +db.connect( + connector: connector, + options: SyncOptions( + crudThrottleTime: Duration(milliseconds: 500), + ), +); +``` + +```kotlin Kotlin +database.connect(connector, crudThrottleMs = 500L) +``` + +```swift Swift +try await database.connect( + connector: connector, + options: ConnectOptions(crudThrottle: 0.5) // seconds +) +``` + +```csharp .NET +await database.Connect(connector, new PowerSyncConnectionOptions( + crudUploadThrottleMs: 500 +)); +``` + + +#### Error Handling + + +If your `uploadData()` throws an error (e.g. due to a `4xx` or `5xx` response from your backend), the SDK will **retry** the same upload indefinitely, effectively blocking the upload queue. Your backend should return `2xx` for validation errors or write conflicts, and reserve error responses for transient failures. See [Writing Client Changes](/handling-writes/writing-client-changes#recommendations) and [Handling Write / Validation Errors](/handling-writes/handling-write-validation-errors) for recommended patterns. + + +When `uploadData()` throws an error: +1. The SDK logs the error and updates the sync status with an `uploadError`. +2. It waits for the retry delay (default: 5 seconds). +3. It retries the upload from the beginning of the queue. +4. If the sync stream disconnects during the retry wait, the upload loop exits and will resume when the connection is re-established. + +#### Stalled Upload Queue Detection + +If the SDK detects that the same CRUD entry is at the front of the queue across consecutive upload iterations (i.e., `uploadData()` returned without error but didn't call `.complete()` on the batch or transaction), it logs a warning: + +> _"Potentially previously uploaded CRUD entries are still present in the upload queue. Make sure to handle uploads and complete CRUD transactions or batches by calling and awaiting their \[.complete()\] method. The next upload iteration will be delayed."_ + +After logging this warning, the SDK **throws an internal error**, which causes it to enter the retry delay path (default: 5 seconds) before attempting the upload again. This prevents a tight loop when `uploadData()` consistently fails to process entries. + +This typically means your `uploadData()` implementation is not calling `.complete()` after successfully processing entries. Always call `.complete()` on the `CrudBatch` or `CrudTransaction` to remove processed entries from the queue. + + +`uploadData()` is only called while the sync stream is connected. If the device is offline, writes accumulate in the upload queue and are uploaded automatically when connectivity is restored and the sync stream reconnects. + + ### When `fetchCredentials()` is Called The PowerSync Client SDK **caches credentials internally** and `fetchCredentials` is called in the following scenarios: diff --git a/handling-writes/writing-client-changes.mdx b/handling-writes/writing-client-changes.mdx index df6a4457..28f7565a 100644 --- a/handling-writes/writing-client-changes.mdx +++ b/handling-writes/writing-client-changes.mdx @@ -4,7 +4,7 @@ description: "Build a backend API endpoint to apply client-side writes from the --- - Your backend application receives the write operations based on how you defined your `uploadData()` function in the `PowerSyncBackendConnector` in your client-side app. See [Client-Side Integration](/configuration/app-backend/client-side-integration) for details. + Your backend application receives the write operations based on how you defined your `uploadData()` function in the `PowerSyncBackendConnector` in your client-side app. See [Client-Side Integration](/configuration/app-backend/client-side-integration) for details, including [when `uploadData()` is called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) and its retry behavior.