Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/jolly-canyons-glow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"nostream": minor
---

perf: added k6 testing for redis on connection and message service rate limiting
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changeset text says this adds k6 testing "for redis". The added scripts exercise relay rate limiting over WebSocket; they don’t directly test Redis behavior, so the release note is misleading. Consider rewording to describe rate limiter load/performance tests without tying it to a specific storage backend.

Suggested change
perf: added k6 testing for redis on connection and message service rate limiting
perf: added k6 testing for connection and message service rate limiting

Copilot uses AI. Check for mistakes.
7 changes: 7 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@
"test:load": "node -r ts-node/register ./scripts/security-load-test.ts",
"smoke:nip03": "node -r ts-node/register scripts/smoke-nip03.ts",
"test:integration": "cucumber-js",
"test:connection": "docker ps | grep nostream > /dev/null && k6 run test/integration/performance/connection-limiting-k6.ts || echo 'Error: nostream container not running'",
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"test:connection": "docker ps | grep nostream > /dev/null && k6 run test/integration/performance/connection-limiting-k6.ts || echo 'Error: nostream container not running'",
"test:performance:connection-rate-limit": "k6 run test/performance/connection-limiting-k6.ts",

"test:message": "docker ps | grep nostream > /dev/null && k6 run test/integration/performance/message-limiting-k6.ts || echo 'Error: nostream container not running'",
Comment thread
cameri marked this conversation as resolved.
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"test:message": "docker ps | grep nostream > /dev/null && k6 run test/integration/performance/message-limiting-k6.ts || echo 'Error: nostream container not running'",
"test:performance:message-rate-limit": "k6 run test/performance/message-limiting-k6.ts",

"cover:integration": "nyc --report-dir .coverage/integration npm run test:integration -- -p cover",
"export": "node --env-file-if-exists=.env -r ts-node/register src/scripts/export-events.ts",
"docker:compose:start": "./scripts/start",
Expand Down Expand Up @@ -100,6 +102,7 @@
"@types/chai-as-promised": "^7.1.5",
"@types/express": "4.17.21",
"@types/js-yaml": "4.0.5",
"@types/k6": "^1.7.0",
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we not need k6 as well? or is it installed globally?

looks like we need to update the docs as well to document this. we can add it to CONTRIBUTING.md

"@types/mocha": "^9.1.1",
"@types/node": "^24.12.2",
"@types/pg": "^8.6.5",
Expand Down
83 changes: 83 additions & 0 deletions test/integration/performance/connection-limiting-k6.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
import { check, sleep } from 'k6';
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's move both files to test/performance

import { Counter } from 'k6/metrics';
import ws from 'k6/ws';

const relayUrl = 'ws://127.0.0.1:8008';
const connectionSuccess = new Counter('connection_success');
const connectionRateLimited = new Counter('connection_rate_limited');

export const options = {
stages: [
{ duration: '10s', target: 3 },
{ duration: '10s', target: 6 },
{ duration: '10s', target: 12 },
{ duration: '10s', target: 18 },
{ duration: '5s', target: 0 },
],
thresholds: {
'ws_connecting': ['p(95)<2000'],
},
};

export default function () {
let socketClosed = false;

const res = ws.connect(relayUrl, {}, function (socket) {
socket.on('close', () => {
socketClosed = true;
connectionRateLimited.add(1);
});

socket.on('open', () => {
connectionSuccess.add(1);
});

socket.setTimeout(() => {
if (!socketClosed) {
socket.close();
}
}, 3000);
Comment on lines +25 to +39
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

connection_rate_limited is incremented on every socket close, including the intentional close triggered by setTimeout. This will misclassify successful connections as rate-limited and also misses true handshake rejections (e.g. HTTP 429), where the close/open handlers never run. Consider classifying rate-limited connections based on the ws.connect() response status (count non-101 responses) and only treating early/abnormal closes as rate limiting if you can reliably identify them (e.g. close code/reason).

Copilot uses AI. Check for mistakes.
});

check(res, {
'status is 101': (r) => r && r.status === 101,
});

sleep(0.5);
}

export function handleSummary(data: any) {
const connSuccess = data.metrics?.connection_success?.values?.count || 0;
const connRateLimited = data.metrics?.connection_rate_limited?.values?.count || 0;
const iterations = data.metrics?.iterations?.values?.count || 0;
const checks = data.metrics?.checks?.values?.passes || 0;
const wsSessions = data.metrics?.ws_sessions?.values?.count || 0;

const totalConnections = connSuccess + connRateLimited;
const successRate = totalConnections > 0 ? ((connSuccess / totalConnections) * 100).toFixed(2) : 0;
const rate = parseFloat(successRate as string);
const successStatus = rate >= 80 ? '✓ GOOD' : rate >= 50 ? '⚠ MODERATE' : '✗ POOR';

Comment thread
cameri marked this conversation as resolved.
console.log(`
╔════════════════════════════════════════════════════════════════╗
║ CONNECTION RATE LIMITER TEST RESULTS ║
╚════════════════════════════════════════════════════════════════╝

EXECUTION:
Iterations: ${iterations}
WebSocket Sessions: ${wsSessions}
Checks Passed: ${checks}

CONNECTIONS:
✓ Success (stayed open): ${connSuccess}
✗ Rate Limited (closed): ${connRateLimited}
─────────────────────
Total: ${totalConnections}

PERFORMANCE:
Success Rate: ${successStatus} ${successRate}%

═══════════════════════════════════════════════════════════════════
`);
return {};
}
106 changes: 106 additions & 0 deletions test/integration/performance/message-limiting-k6.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
import { check } from 'k6';
import { Counter } from 'k6/metrics';
import ws from 'k6/ws';

const relayUrl = 'ws://127.0.0.1:8008';
const noticeCounter = new Counter('notice_messages');
const eoseCounter = new Counter('eose_messages');
const eventCounter = new Counter('event_messages');
const errorCounter = new Counter('error_messages');

export const options = {
stages: [
{ duration: '10s', target: 1 },
{ duration: '10s', target: 2 },
{ duration: '10s', target: 4 },
{ duration: '5s', target: 0 },
],
};

export default function () {
const res = ws.connect(relayUrl, null, function (socket) {
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ws.connect(relayUrl, null, ...): k6's ws.connect expects a params object as the second argument. Passing null is unnecessary and can lead to runtime issues depending on how k6 handles the params value. Prefer {} (or omit the params arg if supported by your k6 version).

Suggested change
const res = ws.connect(relayUrl, null, function (socket) {
const res = ws.connect(relayUrl, {}, function (socket) {

Copilot uses AI. Check for mistakes.
socket.on('open', function () {
let msgCount = 0;
socket.setInterval(function () {
msgCount++;
const text = JSON.stringify(['REQ', `sub-${Date.now()}-${msgCount}`, {limit: 10}]);
socket.send(text);
}, 1000);
});

socket.on('message', function (data) {
try {
const parsed = JSON.parse(data);
const msgType = parsed[0];

if (msgType === 'NOTICE') {
noticeCounter.add(1);
} else if (msgType === 'EOSE') {
eoseCounter.add(1);
} else if (msgType === 'EVENT') {
eventCounter.add(1);
}
} catch (e: any) {
errorCounter.add(1);
console.error('Failed to parse message:', e.message);
}
});

socket.setTimeout(function () {
socket.close();
}, 9000);
});

check(res, {
'status 101': (r) => r && r.status === 101,
});
}

export function handleSummary(data: any) {
const notices = data.metrics?.notice_messages?.values?.count || 0;
const eoses = data.metrics?.eose_messages?.values?.count || 0;
const events = data.metrics?.event_messages?.values?.count || 0;
const iterations = data.metrics?.iterations?.values?.count || 0;
const wsSessions = data.metrics?.ws_sessions?.values?.count || 0;
const msgsSent = data.metrics?.ws_msgs_sent?.values?.count || 0;
const msgsReceived = data.metrics?.ws_msgs_received?.values?.count || 0;
const dataReceived = data.metrics?.data_received?.values?.count || 0;
const checks = data.metrics?.checks?.values?.passes || 0;

const totalMessages = notices + eoses + events;
const successRate = totalMessages > 0 ? ((eoses + events) / totalMessages * 100).toFixed(2) : 0;

const rate = parseFloat(successRate as string);
const successStatus = rate >= 80 ? '✓ GOOD' : rate >= 50 ? '⚠ MODERATE' : '✗ POOR';
const rateLimitStatus = notices > 0 ? '⚠ ACTIVE' : '✓ INACTIVE';
Comment on lines +43 to +75
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file contains TypeScript-only syntax (catch (e: any), data: any, successRate as string). k6 run executes JavaScript and will fail to parse TS annotations unless you add a transpilation step. Either remove TS-only syntax / use JSDoc types and rename to .js, or update the npm scripts to transpile before running k6.

Copilot uses AI. Check for mistakes.

console.log(`
╔════════════════════════════════════════════════════════════════╗
║ MESSAGE RATE LIMITER TEST RESULTS ║
╚════════════════════════════════════════════════════════════════╝

EXECUTION:
Iterations: ${iterations}
WebSocket Sessions: ${wsSessions}
Checks Passed: ${checks}

MESSAGES:
Sent: ${msgsSent}
Received: ${msgsReceived}

MESSAGE TYPES:
✗ NOTICE (rate limited): ${notices}
✓ EOSE (query complete): ${eoses}
◆ EVENT (results): ${events}
─────────────────────
Total: ${totalMessages}

PERFORMANCE:
Success Rate: ${successStatus} ${successRate}%
Data Received: ${dataReceived} bytes
Rate Limiter: ${rateLimitStatus}

═══════════════════════════════════════════════════════════════════
`);
return {};
}
Loading