Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added
- Internal fork of `russh-sftp` as `crates/bssh-russh-sftp` with a `serde_bytes` performance fix for `SSH_FXP_WRITE` and `SSH_FXP_DATA` packets. The upstream serde derive routes `Vec<u8>` through `deserialize_seq` (byte-by-byte), accounting for ~42% of server CPU during 1 GiB SFTP uploads in `perf` profiling. Annotating the `data` fields with `#[serde(with = "serde_bytes")]` and implementing wire-compatible `serialize_bytes` on the SFTP `Serializer` routes through the existing bulk `deserialize_byte_buf`/`try_get_bytes` path. Measured impact on a CPU-bound host (Xeon Silver 4214): 1 GiB SFTP upload throughput improves from 74.8 MiB/s to 96.4 MiB/s (+29%), closing the gap to OpenSSH `sftp-server` from ~26% to ~5%.

### Changed
- Switched the top-level `russh-sftp` dependency from crates.io `russh-sftp = "2.1.1"` to `russh-sftp = { package = "bssh-russh-sftp", version = "2.1.1", path = "crates/bssh-russh-sftp" }`. All existing `use russh_sftp::...` imports continue to work unchanged.

## [2.1.2] - 2026-04-27

### Fixed
Expand Down
46 changes: 28 additions & 18 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
members = [
".",
"crates/bssh-russh",
"crates/bssh-russh-sftp",
]

[package]
Expand All @@ -23,7 +24,8 @@ tokio = { version = "1.52.1", features = ["full"] }
# - Development: uses local path (crates/bssh-russh)
# - Publishing: uses crates.io version (path ignored)
russh = { package = "bssh-russh", version = "0.60.1", path = "crates/bssh-russh" }
russh-sftp = "2.1.1"
# Use our internal russh-sftp fork with a serde_bytes perf fix
russh-sftp = { package = "bssh-russh-sftp", version = "2.1.1", path = "crates/bssh-russh-sftp" }
clap = { version = "4.6.1", features = ["derive", "env"] }
anyhow = "1.0.102"
thiserror = "2.0.18"
Expand Down
35 changes: 35 additions & 0 deletions crates/bssh-russh-sftp/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
[package]
name = "bssh-russh-sftp"
version = "2.1.1"
authors = ["Jeongkyu Shin <inureyes@gmail.com>"]
description = "Temporary fork of russh-sftp with a serde_bytes performance fix for SFTP Write/Data packets"
documentation = "https://docs.rs/bssh-russh-sftp"
edition = "2021"
homepage = "https://github.com/lablup/bssh"
keywords = ["russh", "sftp", "ssh2", "server", "client"]
license = "Apache-2.0"
readme = "README.md"
repository = "https://github.com/lablup/bssh"

[dependencies]
tokio = { version = "1", default-features = false, features = [
"io-util",
"rt",
"sync",
"time",
"macros",
] }
tokio-util = "0.7"
serde = { version = "1.0", features = ["derive"] }
serde_bytes = "0.11"
bitflags = { version = "2.9", features = ["serde"] }
async-trait = { version = "0.1", optional = true }

thiserror = "2.0"
chrono = "0.4"
bytes = "1.10"
log = "0.4"
flurry = "0.5"

[features]
async-trait = ["dep:async-trait"]
26 changes: 26 additions & 0 deletions crates/bssh-russh-sftp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# bssh-russh-sftp

Temporary fork of [russh-sftp](https://crates.io/crates/russh-sftp) with a `serde_bytes` performance fix for SFTP `Write` and `Data` packets.

This crate exists so bssh can ship the packet serialization fix independently while keeping the public crate name usable through Cargo's `package = "bssh-russh-sftp"` dependency alias.

## The Problem

`russh-sftp` 2.1.1 derives serde for `Vec<u8>` fields in `SSH_FXP_WRITE` and `SSH_FXP_DATA`. With the crate's custom deserializer, that routes through `deserialize_seq` and reads payload bytes one at a time. Large transfers spend substantial CPU in serde's generic `VecVisitor` path.

## The Fix

The fork annotates the binary payload fields with `#[serde(with = "serde_bytes")]` and implements compatible `serialize_bytes` framing in the SFTP serializer. The wire format remains `u32 length + bytes`, but deserialization uses the existing bulk byte-buffer path.

## Sync with Upstream

```bash
cd crates/bssh-russh-sftp
./sync-upstream.sh 2.1.1
```

Local changes are kept as patch files under `patches/`.

## License

Apache-2.0 (same as russh-sftp)
48 changes: 48 additions & 0 deletions crates/bssh-russh-sftp/create-patch.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
#!/bin/bash
# create-patch.sh
# Creates a patch file from the current bssh-russh-sftp changes compared to upstream russh-sftp.
#
# Usage: ./create-patch.sh

set -e

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
BSSH_ROOT="$SCRIPT_DIR/../.."
UPSTREAM_DIR="$BSSH_ROOT/references/russh-sftp/src"
CURRENT_DIR="$SCRIPT_DIR/src"
PATCH_DIR="$SCRIPT_DIR/patches"
PATCH_FILE="$PATCH_DIR/sftp-serde-bytes-perf.patch"

GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'

log_info() { echo -e "${GREEN}[INFO]${NC} $1"; }
log_warn() { echo -e "${YELLOW}[WARN]${NC} $1"; }

if [ ! -d "$UPSTREAM_DIR" ]; then
echo "Error: Upstream russh-sftp not found at $UPSTREAM_DIR"
echo "Please ensure references/russh-sftp exists with the upstream source."
exit 1
fi

mkdir -p "$PATCH_DIR"

log_info "Creating patch from differences..."

/usr/bin/diff -urN "$UPSTREAM_DIR" "$CURRENT_DIR" \
| sed "s|$UPSTREAM_DIR|a/src|g" \
| sed "s|$CURRENT_DIR|b/src|g" \
> "$PATCH_FILE" || true

if [ -s "$PATCH_FILE" ]; then
LINES=$(wc -l < "$PATCH_FILE" | tr -d ' ')
log_info "Patch created: $PATCH_FILE ($LINES lines)"

echo ""
echo "Patch summary:"
echo "=============="
grep -E "^@@|^\+\+\+|^---" "$PATCH_FILE" | head -20
else
log_warn "No differences found - patch file is empty"
fi
Loading