feat(integrations): support custom request headers on MCP shttp/sse transports

[neubig]

This PR was created by an AI agent (OpenHands) on behalf of @neubig.

Closes #363.

Summary

Adds headerFields?: MarketplaceField[] to the shttp / sse variants of IntegrationTransport, mirroring the envFields/argFields pattern on stdio. This lets a hosted MCP integration declare a list of named request headers the user must supply — the generalized mechanism behind header-based auth flows like Datadog's DD-API-KEY / DD-APPLICATION-KEY (https://docs.datadoghq.com/mcp_server/setup/?tab=other#authentication), which were previously expressible only via OAuth.

headerFields composes with the existing api_key/bearer/basic/oauth2 strategies (the Bearer token from api_key is folded into Authorization separately) — no new auth-strategy enum value is needed.

Changes

• integrations/index.d.ts: add headerFields?: MarketplaceField[] to shttp and sse transports.
• integrations/catalog/datadog.json: add a non-OAuth api connection option declaring DD-API-KEY and DD-APPLICATION-KEY as headerFields, so the marketplace install flow can surface the header-based path without Datadog Technology Partner Program enrollment.
• tests/test_catalogs.py:
  • test_remote_no_auth_mcp_entries_are_intentionally_public tightened to exclude options with headerFields (they still require credentials, just via named headers, so they are not "public/no-auth").
  • test_credential_fields_have_helper_text_and_link extended to validate headerFields password fields too.

Why this is general

• Any MCP shttp/sse integration can declare the headers it needs — not a Datadog-specific hack.
• The matching frontend collection UI is tracked separately on agent-canvas (MCP: collect custom request headers in install modal + custom editor (e.g. Datadog) agent-canvas#1501). The agent-canvas plumbing (MCPServerConfig.headers, MCPSSEServer.headers, mcp-service.api.ts, use-update-mcp-server.ts) and the backend (software-agent-sdk mcp_router._RemoteMCPServerSpec.headers, fastmcp transport, encrypted at rest by settings/model.py) already support arbitrary headers end-to-end.

Test plan

• uv run --group test pytest -q — 364 passed
• npm run build:integrations — catalog-index.js unchanged (datadog.json already imported)
• CI green

[all-hands-bot]

✅ Review complete.

This review was performed through OpenHands Cloud Automation. You can log in and view the conversation here.

[all-hands-bot]

🟢 Good taste — Elegant, minimal extension that solves a real problem cleanly.

Summary

This PR adds headerFields?: MarketplaceField[] to the shttp and sse transport variants in IntegrationTransport, directly mirroring the envFields/argFields pattern already established on stdio. The Datadog catalog entry is a solid first consumer, properly wiring up DD-API-KEY and DD-APPLICATION-KEY for users who cannot or do not want to go through the Datadog Technology Partner OAuth flow.

What makes this good:

• Pattern conformance: The new field is a direct analog of envFields on stdio — no new concepts, just a consistent extension of an existing idiom to a different transport kind.
• Design choice is sound: Using auth.strategy: "none" with headerFields keeps the auth strategy enum focused on how OpenHands manages credentials internally (Bearer injection, storage, etc.), while headerFields handles user-supplied transport-level headers. The PR description justifies this clearly, and the test has been correctly updated to prevent strategy: "none" + headerFields entries from being silently classified as "public/no-auth".
• Test coverage: Both affected tests are updated correctly. test_remote_no_auth_mcp_entries_are_intentionally_public now excludes header-credentialed options from the public-access guard, and test_credential_fields_have_helper_text_and_link correctly validates headerFields password fields alongside envFields and argFields. All 5 tests pass.
• Backward compatible: Only optional fields are added; no existing catalog entry or consumer is affected.
• Documentation: The JSDoc on shttp.headerFields captures genuinely non-obvious behavior — the composability with auth strategies and the Authorization header conflict warning are useful for future catalog authors.

[RISK ASSESSMENT]

• [Overall PR] ⚠️ Risk Assessment: 🟢 LOW
  Catalog type extension following an established pattern (envFields → headerFields). No auth logic changed; blast radius is limited to new headerFields consumers. The updated tests guard correctly against misclassification as public/no-auth. Safe to merge.

VERDICT:
✅ Worth merging: Clean, pattern-consistent implementation with appropriate test coverage.

KEY INSIGHT:
headerFields is the natural transport-layer analog of envFields, not a new auth concept — keeping the concern separation clean.

This review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation. View conversation

[openhands-release-bot]

🚀 Released in v0.9.0.