Durga

BPMN-driven Kafka scaffolding tool. Reads a BPMN model and generates Kafka-oriented worker, gateway, orchestration, and topic setup skeletons for process implementations.

System manual | BPMN coverage matrix | Beta support boundary | Maturity plan | Release checklist | Operations hardening | Deployment guide | Plugin architecture | Testcontainers setup

Quick start

mvn -q clean package
java -jar target/durga-0.1.0-beta.1.jar path/to/process.bpmn

Local Kafka setup

cd setup && docker compose up

Kafka UI will log transient connection errors until the broker is ready — this is normal. The broker listens on localhost:9094.

Running tests

# Unit tests and generated-project checks (no Docker):
mvn test -Dtest='!*IntegrationTest'

# Monitoring UI tests and build:
cd monitoring-ui && npm ci && npm test && npm run build

# Integration tests (require Docker):
mvn test -Dtest='*IntegrationTest'

# If Docker detection fails, use the Linux container fallback:
./setup/run-integration-tests.sh

See Testcontainers setup for details.

BPMN scaffolding

Output lands in generated/ by default:

• Java sources in generated/src/main/java/org/gautelis/durga/generated/
• topics.sh and summary.json
• task-payloads.json with sample input payloads
• pom.xml and README.md for the generated project
• Helper scripts: demo-scenario.sh, send-task-input.sh, complete-task.sh, fail-task.sh, escalate-task.sh, complete-call-activity.sh, send-message-event.sh, send-signal-event.sh, watch-process-events.sh, watch-task-output.sh

Flags:

• --dry-run — print summary.json, topics.sh, connect configs, and application YAML without writing files
• --out <dir> — custom output directory
• --event-topic <topic> — override the canonical lifecycle event topic (default: process-events-{processId}). Each pipeline gets an isolated topic by default; use this flag to share a topic across pipelines or use a custom name.
• --transactions — generate transactional workers using Kafka producer/consumer APIs

The generator skips existing files in src/main/java/, merges new channels into application.yml, and evaluates gateway conditions from BPMN conditionExpression at runtime.

BPMN sample catalog

All sample models live under src/test/resources/bpmn/. Run any with:

mvn -q clean package
java -jar target/durga-0.1.0-beta.1.jar src/test/resources/bpmn/<model>.bpmn

Model	Feature
invoice_receipt.bpmn	Baseline process (start, service, review, approve/reject, notify)
order_fulfillment.bpmn	Legacy reference model
invoice_receipt_reminder.bpmn	Intermediate timer catch
invoice_message_exchange.bpmn	Message throw/catch
invoice_signal_exchange.bpmn	Signal throw/catch
invoice_review_deadline.bpmn	Interrupting timer boundary
invoice_review_reminder_non_interrupting.bpmn	Non-interrupting timer boundary
invoice_processing_error.bpmn	Interrupting error boundary
invoice_review_escalation.bpmn	Interrupting escalation boundary
invoice_call_activity.bpmn	Call activity request/reply
invoice_review_subprocess.bpmn	Embedded subprocess
invoice_nested_subprocess.bpmn	Nested subprocesses
invoice_subprocess_deadline.bpmn	Interrupting timer boundary on subprocess
invoice_subprocess_reminder_non_interrupting.bpmn	Non-interrupting timer boundary on subprocess
invoice_subprocess_error.bpmn	Interrupting error boundary on subprocess
invoice_event_subprocess_message.bpmn	Non-interrupting message-start event subprocess
invoice_event_subprocess_interrupting_message.bpmn	Interrupting message-start event subprocess
invoice_event_subprocess_timer.bpmn	Timer-start event subprocess
invoice_event_subprocess_error.bpmn	Error-start event subprocess
data_pipeline_demo.bpmn	Plugin-annotated pipeline (json-transform, field-filter, kv-enricher)
order_events_pipeline.bpmn	8-plugin order pipeline with XOR gateway; use --connect for source/sink
log_processing_pipeline.bpmn	Regex, template, flatten, validate, mask; use --connect
custom_activity_demo.bpmn	Custom activity with contract interface + delegating worker

Monitoring

A Kafka Streams topology consumes per-process lifecycle events and materializes:

• latest state per instance into process-state-{processId}
• counts by state into process-state-counts-{processId}
• active-instance index into process-active-state-{processId}
• activity latency summaries into process-latency-{processId}
• coarse lifecycle trend buckets into process-trends-{processId}

Each monitoring instance is scoped to a single processId. Topics and state stores are suffixed with -{processId} so multiple monitors can coexist.

Full-stack dev demo (one command)

./setup/dev-up.sh

Starts everything — Kafka in Docker, the monitoring backend, a continuous process-feed generator, and the Svelte SPA via Vite dev server (hot-reload). Opens the dashboard at http://localhost:5173. Press Ctrl+C to stop all services.

Two processes side by side (two browser tabs):

PROCESSES="invoice_receipt:src/test/resources/bpmn/invoice_receipt.bpmn,order_fulfillment:src/test/resources/bpmn/order_fulfillment.bpmn" \
  ./setup/dev-up.sh

Backends land on adjacent ports: :8081/:5173 for the first process, :8082/:5174 for the second.

Fast restart (skip the Maven + npm builds):

SKIP_BUILD=true ./setup/dev-up.sh

Configurable environment variables:

Variable	Default	Purpose
PROCESSES	invoice_receipt:src/test/resources/bpmn/invoice_receipt.bpmn:1000	Comma-separated pid:bpmn[:intervalMs]
BOOTSTRAP	localhost:9094	Kafka bootstrap servers
BACKEND_PORT	8081	Base backend API port
VITE_PORT	5173	Base Vite dev-server port
START_KAFKA	true	Auto-start Kafka via Docker Compose
SKIP_BUILD	false	Skip mvn package -DskipTests and npm install

Manual dev setup (step by step)

The same as dev-up.sh, broken into separate terminals for debugging:

# Terminal 1 — Kafka
cd setup && docker compose up -d

# Build
mvn -q package -DskipTests

# Terminal 2 — Monitoring backend (API + BPMN diagram)
java -cp target/durga-0.1.0-beta.1.jar \
  org.gautelis.durga.monitoring.MonitoringContainer \
  localhost:9094 durga-monitoring 8081 invoice_receipt \
  src/test/resources/bpmn/invoice_receipt.bpmn

# Terminal 3 — Svelte SPA (Vite dev server, proxies /api → :8081)
cd monitoring-ui && npm run dev

# Terminal 4 — Continuous process feed
java -cp target/durga-0.1.0-beta.1.jar \
  org.gautelis.durga.demo.ContinuousFeedPublisher \
  localhost:9094 invoice_receipt 1000

Open http://localhost:5173 for the dashboard. The SPA displays live counts, latency stats, stuck instances, lifecycle trends, and a BPMN diagram with color-coded activity overlays.

HTTP API endpoints

All paths are available with or without the /api/ prefix:

• GET /health — Kafka Streams state (RUNNING, REBALANCING, etc.)
• GET /dashboard — minimal HTML dashboard (for direct browser access)
• GET /instances/{processInstanceId} — latest state view for one instance
• GET /processes/{processId}/counts — state counts per process
• GET /processes/{processId}/latency — per-activity latency summaries
• GET /processes/{processId}/trends — lifecycle trend buckets
• GET /counts — counts across all monitored processes
• GET /stuck?processId=<id>&olderThanSeconds=60 — stuck-instance detection
• GET /diagram — BPMN 2.0 XML (if a diagram file was provided at startup)
• GET /metrics — Micrometer metrics in Prometheus text format

CLI client

java -cp target/durga-0.1.0-beta.1.jar \
  org.gautelis.durga.monitoring.ProcessMonitoringClient \
  http://localhost:8081 health

java -cp target/durga-0.1.0-beta.1.jar \
  org.gautelis.durga.monitoring.ProcessMonitoringClient \
  http://localhost:8081 counts invoice_receipt

java -cp target/durga-0.1.0-beta.1.jar \
  org.gautelis.durga.monitoring.ProcessMonitoringClient \
  http://localhost:8081 latency invoice_receipt

java -cp target/durga-0.1.0-beta.1.jar \
  org.gautelis.durga.monitoring.ProcessMonitoringClient \
  http://localhost:8081 stuck invoice_receipt 60

java -cp target/durga-0.1.0-beta.1.jar \
  org.gautelis.durga.monitoring.ProcessMonitoringClient \
  http://localhost:8081 instance <processInstanceId>

Demo scenarios (without a generated process)

java -cp target/durga-0.1.0-beta.1.jar \
  org.gautelis.durga.demo.ProcessEventScenarioRunner \
  localhost:9094 happy invoice_receipt register_invoice,review_invoice,notify_requester

# Also available: stuck, failed

CLI demo script (start Kafka, publish scenario, query)

./setup/demo-monitoring.sh
START_KAFKA=true ./setup/demo-monitoring.sh
SCENARIO=stuck ./setup/demo-monitoring.sh
SCENARIO=failed ./setup/demo-monitoring.sh
BPMN_PATH=src/test/resources/bpmn/invoice_receipt.bpmn ./setup/demo-monitoring.sh

Docker demo (continuous feed + monitoring backend)

docker compose -f setup/docker-compose.demo.yml up --build

Starts Kafka, the monitoring backend, and a continuous feed publisher. Open http://localhost:8081 for the API, http://localhost:8080 for Kafka UI. For the Svelte SPA, run cd monitoring-ui && npm run dev and open http://localhost:5173 (Vite proxies /api to :8081).

# Customise feed parameters
FEED_PROCESS_ID=order_fulfillment FEED_INTERVAL_MS=2000 \
  docker compose -f setup/docker-compose.demo.yml up --build

Generated project probes

Every scaffolded project includes producer and observer helpers:

./generated/demo-scenario.sh happy
./generated/send-task-input.sh register_invoice
./generated/complete-task.sh approve_invoice <instance-id>
./generated/fail-task.sh register_invoice <instance-id>
./generated/escalate-task.sh review_invoice <instance-id>
./generated/complete-call-activity.sh validate_invoice_process <instance-id>
./generated/send-message-event.sh invoice_review_response_message <instance-id>
./generated/send-signal-event.sh invoice_review_signal_signal <instance-id>
./generated/watch-process-events.sh
./generated/watch-task-output.sh register_invoice

Embedded subprocesses generate scope entry/completion services. Event subprocesses with message or signal starts generate start/completion services driven by external Kafka topics. Interrupting starts emit cancellation for the enclosing scope; non-interrupting starts branch alongside the parent flow. Timer, error, and escalation event subprocess starts are supported within embedded subprocesses.

User interfaces

Processes (or data pipelines) are managmed using any BPMN modeler, such as Camunda Modeler.

The monitoring tool also uses the BPMN model as a backdrop to presenting statistics.