Skip to content

feat(lora-ingest): ChirpStack MQTT subscriber + krobjob relay service#2

Closed
Runeov wants to merge 2 commits into
mainfrom
feat/lora-ingest-service
Closed

feat(lora-ingest): ChirpStack MQTT subscriber + krobjob relay service#2
Runeov wants to merge 2 commits into
mainfrom
feat/lora-ingest-service

Conversation

@Runeov

@Runeov Runeov commented Jun 17, 2026

Copy link
Copy Markdown
Owner

Summary

  • New lora-ingest/ TypeScript service: subscribes to ChirpStack MQTT, decrypts AES-256-GCM app-layer payload, relays to krobjob
  • decoder.ts: AES-256-GCM decrypt + fPort 1 (sale, 7B) and fPort 2 (heartbeat, 3B) parser
  • relay.ts: HMAC-SHA256 signed fetch to krobjob /api/internal/lora-ingest
  • index.ts: MQTT connect/subscribe/reconnect loop
  • Zero extra runtime dependencies (mqtt@5 + Node built-in crypto)

Test plan

  • Copy .env.example to .env, fill APP_ENC_KEY_HEX and BASEFLOW_HMAC_KEY
  • npm run dev against a local ChirpStack instance
  • Send a test uplink from lora_node.py and verify krobjob receives the POST
  • Confirm relay logs no errors and krobjob returns 200

Dependencies

  • APP_ENC_KEY_HEX must match APP_ENC_KEY_HEX in pilot/lora-node/lora_node.py
  • BASEFLOW_HMAC_KEY must match BASEFLOW_HMAC_KEY Vercel env var on krobjob
  • Companion PR: Runeov/skeleton#45

Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added a LoRa ingest service that connects to a ChirpStack MQTT broker and processes uplink messages.
    • Validates the encryption key and decrypts payloads, then strictly decodes event data (sale/heartbeat).
    • Relays decoded events to the configured endpoint using signed HTTP requests, retrying on failures and including signal quality, gateway info, and timestamps.
  • Chores
    • Added a full Node/TypeScript (ESM) project setup, including build/run scripts, configuration, and an environment variable template.

New Node.js/TypeScript service in lora-ingest/:
- Subscribes to ChirpStack application MQTT uplinks
- Decrypts AES-256-GCM app-layer payload (matches lora_node.py encoder)
- Routes fPort 1 (sale, 7B) and fPort 2 (heartbeat, 3B)
- Signs relay POST with HMAC-SHA256 and forwards to krobjob ingest API
TypeScript clean, mqtt@5, zero extra runtime dependencies.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@vercel

vercel Bot commented Jun 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
baseflow Ready Ready Preview, Comment Jun 17, 2026 4:35am

@coderabbitai

coderabbitai Bot commented Jun 17, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 21f77331-8364-4728-99e2-ac73d215db42

📥 Commits

Reviewing files that changed from the base of the PR and between 4da6419 and fadba64.

📒 Files selected for processing (3)
  • lora-ingest/src/decoder.ts
  • lora-ingest/src/index.ts
  • lora-ingest/src/relay.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • lora-ingest/src/relay.ts
  • lora-ingest/src/index.ts

📝 Walkthrough

Walkthrough

A new lora-ingest Node.js/TypeScript package is introduced from scratch. It connects to a ChirpStack MQTT broker, receives LoRa uplink messages, decrypts AES-256-GCM base64 payloads, decodes binary events by fPort (sale or heartbeat), and forwards HMAC-SHA256–signed JSON to an internal HTTP ingest endpoint.

Changes

LoRa Ingest Service

Layer / File(s) Summary
Project scaffolding and environment config
lora-ingest/package.json, lora-ingest/tsconfig.json, lora-ingest/.env.example
Defines the ESM Node.js project with mqtt as the runtime dependency, TypeScript compiler options targeting ES2022/NodeNext with output to dist/, and all required environment variable placeholders with inline documentation.
AES-256-GCM decryption and fPort event decoding
lora-ingest/src/decoder.ts
Exports the DecodedEvent union (sale, heartbeat), decryptPayload which parses a 12-byte nonce + ciphertext + 16-byte auth tag envelope and decrypts with APP_ENC_KEY_HEX, and decodeEvent which reads big-endian fields from plaintext keyed by fPort (1 = sale, 7 bytes; 2 = heartbeat, 3 bytes), returning null on auth failure, unsupported port, or insufficient length.
HMAC-signed HTTP relay
lora-ingest/src/relay.ts
Exports RelayPayload (extending DecodedEvent with rssi, snr, gatewayEui, timestamp) and relay(), which JSON-stringifies the payload, computes an HMAC-SHA256 signature using BASEFLOW_HMAC_KEY, and POSTs to KROBJOB_INGEST_URL with X-Baseflow-Signature, throwing an error with HTTP status and response body on failure.
MQTT ingestion entrypoint
lora-ingest/src/index.ts
Connects to the ChirpStack MQTT broker via CHIRPSTACK_MQTT_URL, subscribes to the application uplink topic, and for each message validates required fPort/data fields, runs the decrypt → decode → relay pipeline with RSSI/SNR/gatewayEui/timestamp enrichment (defaulting missing metadata), retries relay with exponential backoff over 3 attempts, and handles MQTT lifecycle events (error, reconnect, disconnect).

Sequence Diagram(s)

sequenceDiagram
  participant ChirpStack as ChirpStack MQTT Broker
  participant index as index.ts (MQTT client)
  participant decoder as decoder.ts
  participant relay as relay.ts
  participant ingest as KROBJOB_INGEST_URL

  ChirpStack->>index: MQTT uplink message (JSON with fPort, data)
  index->>index: JSON parse + required-field guard
  index->>decoder: decryptPayload(data: base64)
  decoder-->>index: plaintext Buffer or null
  index->>decoder: decodeEvent(fPort, plaintext)
  decoder-->>index: DecodedEvent or null
  index->>relay: relayWithRetry(DecodedEvent + rssi/snr/gatewayEui/timestamp)
  relay->>relay: HMAC-SHA256 sign JSON body (BASEFLOW_HMAC_KEY)
  relay->>ingest: POST /ingest with X-Baseflow-Signature
  ingest-->>relay: HTTP response
  relay-->>index: resolve or throw on non-OK
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐇 A rabbit hops through radio waves so small,
AES keys unlock the signal's call,
Big-endian bytes tell tales of a sale,
HMAC-signed, the data sets sail,
From fPort to ingest, no packet shall stall!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 28.57% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly and accurately summarizes the main changes: introducing a ChirpStack MQTT subscriber service that relays data to krobjob, which matches the PR's core purpose of connecting MQTT ingestion with relay functionality.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/lora-ingest-service

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@lora-ingest/src/decoder.ts`:
- Around line 30-45: The payload length validation in the fPort decoding logic
uses >= operators instead of exact equality checks, which allows
trailing/malformed data to pass validation. Replace the >= operators with ===
operators in both the fPort === 1 and fPort === 2 conditions to enforce strict
fixed-width validation. The sale event handler should check plaintext.length ===
7 (not >=), and the heartbeat event handler should check plaintext.length === 3
(not >=) to reject payloads with unexpected extra bytes.
- Line 3: The APP_ENC_KEY initialization needs to validate the APP_ENC_KEY_HEX
environment variable at startup to catch configuration errors early. Add
validation to check that the environment variable is present, contains valid
hexadecimal characters, and decodes to exactly 32 bytes. If any validation
fails, throw an error or exit the process immediately with a clear error
message. This should be done before or during the assignment to APP_ENC_KEY to
prevent silent decrypt failures downstream that would be difficult to diagnose.

In `@lora-ingest/src/index.ts`:
- Around line 47-57: The relay() function call in the try-catch block currently
logs failures and discards the event without retrying, causing permanent data
loss during transient HTTP failures. Replace the direct relay() call with a
retry wrapper function that implements bounded retry logic with exponential or
linear backoff between attempts, limiting retries to a reasonable number (e.g.,
3-5 attempts) with increasing delays between each attempt, and only log and
discard the event if all retry attempts are exhausted.
- Around line 5-8: The non-null assertions on CHIRPSTACK_MQTT_URL and
CHIRPSTACK_APPLICATION_ID environment variables mask missing configuration
errors. Before creating the MQTT client connection with mqtt.connect, add
explicit validation checks for both CHIRPSTACK_MQTT_URL and
CHIRPSTACK_APPLICATION_ID to ensure they exist, throwing a descriptive error if
either is undefined or empty. This validation should occur early in the
bootstrap sequence, before the mqtt.connect call is attempted, so configuration
issues are caught immediately rather than causing unexpected runtime behavior.

In `@lora-ingest/src/relay.ts`:
- Around line 4-6: Add validation logic immediately after the KROBJOB_INGEST_URL
and BASEFLOW_HMAC_KEY constant declarations to ensure both environment variables
are present and non-empty. Additionally, validate that KROBJOB_INGEST_URL is a
properly formatted URL by attempting to parse it with the URL constructor. If
either variable is missing, empty, or if URL parsing fails, throw an error with
a descriptive message to fail fast at module initialization rather than silently
failing during runtime relay operations.
- Around line 18-25: The fetch call to KROBJOB_INGEST_URL in the relay operation
lacks a request timeout, causing requests to hang for up to 300 seconds by
default which can lead to resource exhaustion during outages. Add an AbortSignal
with a 10-second timeout to the fetch options by including a signal property set
to AbortSignal.timeout(10_000) in the fetch configuration object alongside the
existing method, headers, and body properties.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1c896a9a-3520-46e3-8eb9-8e269da073e0

📥 Commits

Reviewing files that changed from the base of the PR and between 7d1fdf8 and 4da6419.

⛔ Files ignored due to path filters (1)
  • lora-ingest/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (6)
  • lora-ingest/.env.example
  • lora-ingest/package.json
  • lora-ingest/src/decoder.ts
  • lora-ingest/src/index.ts
  • lora-ingest/src/relay.ts
  • lora-ingest/tsconfig.json

Comment thread lora-ingest/src/decoder.ts Outdated
Comment thread lora-ingest/src/decoder.ts Outdated
Comment thread lora-ingest/src/index.ts Outdated
Comment thread lora-ingest/src/index.ts
Comment thread lora-ingest/src/relay.ts Outdated
Comment thread lora-ingest/src/relay.ts
decoder.ts: requireHexKey validates APP_ENC_KEY_HEX at boot (present,
hex-only, exactly 32 bytes); payload length checks changed >= to ===
for strict fixed-width protocol enforcement.

relay.ts: requireEnv + URL constructor validate KROBJOB_INGEST_URL and
BASEFLOW_HMAC_KEY at startup; AbortSignal.timeout(10_000) prevents
300-second default hang on outbound fetch.

index.ts: requireEnv validates CHIRPSTACK_MQTT_URL and
CHIRPSTACK_APPLICATION_ID before connect; relayWithRetry wraps relay()
with 3 attempts + exponential backoff (250ms, 500ms) to survive
transient HTTP failures without permanent uplink loss.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@Runeov

Runeov commented Jun 17, 2026

Copy link
Copy Markdown
Owner Author

Parking this — superseded, not merging. The N100/ChirpStack host this relay targeted is gone, and the LoRa layer was re-scoped (2026-06-17) from a bespoke LoRaWAN+ChirpStack telemetry pipeline to a Meshtastic coordination layer (compute centralized into one solar/compute hub). Branch feat/lora-ingest-service is left intact as a frozen, revivable reference. To revive: see project-kj pilot/heltec-node/PARKED.md (pick a host = solar hub or cloud, not the N100).

@Runeov Runeov closed this Jun 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant