feat(sidecar): configurable logfmt logging + ext stall-watchdog tracing#1517
Closed
NathanFlurry wants to merge 1 commit into
Closed
feat(sidecar): configurable logfmt logging + ext stall-watchdog tracing#1517NathanFlurry wants to merge 1 commit into
NathanFlurry wants to merge 1 commit into
Conversation
Replace the hardcoded ERROR-level tracing subscriber in agentos-sidecar with a rivet-style init_tracing: EnvFilter-gated level (AGENTOS_LOG_LEVEL > LOG_LEVEL > RUST_LOG > info), logfmt format (RUST_LOG_FORMAT=text to opt out), and an optional file sink (AGENTOS_LOG_FILE). Output stays on stderr/file, never stdout (the frame channel). Add an ext.request span + entry/exit timing in acp_extension, plus a stall watchdog that warns every 10s while a request is in flight, so a hung tool turn surfaces as a breadcrumb long before the host's 120s frame timeout instead of silently. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
railway-app
Bot
temporarily deployed
to
rivet-frontend / agentos-pr-1517
|
🚅 Deployed to the agentos-pr-1517 environment in agentos
🚅 Deployed to the agentos-pr-1517 environment in rivet-frontend
|
Member
Author
|
Superseded by the combined PR #1518 (docs + options-schema + sidecar/host-tools-zod together). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Make the
agentos-sidecarlogging configurable (matching the~/rivetsetup) and add the tracing needed to see where anextrequest — especially a tool-call turn — spends its time or hangs.1. Logging setup (mirrors rivet's
init_tracing)crates/agentos-sidecar/src/main.rspreviously hardcodedLevel::ERRORwith noEnvFilter, soRUST_LOGwas silently ignored and the sidecar was effectively unobservable. Now:AGENTOS_LOG_LEVEL>LOG_LEVEL>RUST_LOG>info.tracing-logfmt),RUST_LOG_FORMAT=textto opt out.AGENTOS_LOG_FILEwrites to a file; otherwise stderr. Never stdout (that carries the binary frame protocol).RUST_LOG_{SPAN_NAME,SPAN_PATH,TARGET,LOCATION,MODULE_PATH,ANSI_COLOR}.2. ext-request tracing + stall watchdog
crates/agentos-sidecar/src/acp_extension.rs: wraphandle_payloadin anext.requestspan withkind+elapsed_ms, emit received/handled events, and — the key bit — a stall watchdog thatwarn!s every 10s while a request is still in flight. A hung tool turn now leaves a breadcrumb long before the host's 120s frame timeout instead of timing out silently.Why
Debugging a Zid tool-call hang (
timed out waiting for sidecar protocol frame for ext), we found the sidecar emitted nothing: ERROR-only,RUST_LOG-deaf, and no per-request timing. This restores parity with rivet logging and makes the hang (and session-creation latency) diagnosable.Notes
info; prod behavior is unchanged unless a level is set.binding.invoketracing on the tool-resolution path (separate PR; independent to compile).cargo check -p agentos-sidecarpasses.🤖 Generated with Claude Code