Skip to content

Migrate Valence Security Sentinel connector to CCF Push (Codeless Connector Framework)#14677

Open
shlomimatichin wants to merge 1 commit into
Azure:masterfrom
valencesec:valence-ccf-push-migration
Open

Migrate Valence Security Sentinel connector to CCF Push (Codeless Connector Framework)#14677
shlomimatichin wants to merge 1 commit into
Azure:masterfrom
valencesec:valence-ccf-push-migration

Conversation

@shlomimatichin

Copy link
Copy Markdown
Contributor

Migrate Valence Security Sentinel connector to CCF Push (Codeless Connector Framework)

Replaces the deprecated Log Analytics HTTP Data Collector API (Workspace ID + Primary Key) connector with a CCF Push connector using DCE/DCR-based auth via the Azure Monitor Logs Ingestion API. This follows Microsoft's deprecation of the sharedKeys-based connector pattern.

Note: This PR supersedes #14528. That PR was opened from the fork's master branch, which caused the DetectionsValidations / idChangeValidator check to fail (head and base were both named master, so the validator tried to fetch the fork's master into the local master ref and hit a non-fast-forward rejection). This PR is the same content, rebased onto the latest master and submitted from a dedicated feature branch. #14528 will be closed.

Changes:

  • DELETE: ValenceSecurity.json (legacy Data Collector API connector)
  • ADD: ValenceSecurity_PUSH_CCP/ folder with 5 CCF files:
    • ValenceSecurity_connectorDefinition.json (kind: Customizable)
    • ValenceSecurity_PollerConfig.json (kind: Push)
    • ValenceSecurity_DCR.json (two streams: ValenceAlerts + ValenceAuditLogs)
    • ValenceSecurity_table_ValenceAlerts.json
    • ValenceSecurity_table_ValenceAuditLogs.json
  • UPDATE: Analytic rule to reference new table (ValenceAlerts_CL) and CCF connector ID (ValenceSecurityCcfPush); field names updated to CCF schema
  • UPDATE: Workbook to use ValenceAlerts_CL and new severity field name
  • UPDATE: Solution data file to point to CCF connector definition; v3.0.3
  • REGENERATE: Package (mainTemplate.json, createUiDefinition.json, testParameters.json, 3.0.3.zip) using the V3 packaging tool
  • ADD: ValenceSecurityCcfPush to ValidConnectorIds.json (merged with latest master)

Testing Completed: Yes — validated in a Microsoft Sentinel environment; KQL and YAML validation run locally.

…nector Framework)

Migrates the Valence Security data connector to the Codeless Connector
Framework (CCF) Push model: adds the DCR, poller config, connector
definition, and ValenceAlerts/ValenceAuditLogs table definitions, bumps
the solution package to 3.0.3, and registers ValenceSecurityCcfPush as a
valid connector id.

Resubmitted from a feature branch (rebased onto latest master) to replace
PR Azure#14528, which was opened from the fork's master branch and tripped the
idChangeValidator (head/base both named "master").

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Migrates the Valence Security Microsoft Sentinel connector from the legacy Log Analytics HTTP Data Collector (shared keys) pattern to a CCF Push connector using DCE/DCR + Azure Monitor Logs Ingestion API.

Changes:

  • Replaces the legacy connector with a new ValenceSecurity_PUSH_CCP/ CCF Push connector definition, DCR, poller config, and table schemas.
  • Updates analytic rule + workbook queries to use the new ValenceAlerts_CL schema and connector ID ValenceSecurityCcfPush.
  • Regenerates solution package artifacts and updates solution versioning and validation connector IDs.

Reviewed changes

Copilot reviewed 14 out of 16 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
Solutions/Valence Security/Workbooks/ValenceAlertsWorkbook.json Updates workbook KQL to reference ValenceAlerts_CL and new severity field.
Solutions/Valence Security/Package/testParameters.json Adds/updates test parameters used by the regenerated ARM package.
Solutions/Valence Security/Package/mainTemplate.json Major packaging update: new CCF Push connector resources, DCR/table deployment, updated versions/IDs.
Solutions/Valence Security/Package/createUiDefinition.json Updates installer text/formatting and connector UI entries.
Solutions/Valence Security/Data/Solution_ValenceSecurity.json Points solution data to the new CCF connector definition and bumps version.
Solutions/Valence Security/Data Connectors/ValenceSecurity_PUSH_CCP/ValenceSecurity_table_ValenceAuditLogs.json Adds audit logs custom table schema definition.
Solutions/Valence Security/Data Connectors/ValenceSecurity_PUSH_CCP/ValenceSecurity_table_ValenceAlerts.json Adds alerts custom table schema definition.
Solutions/Valence Security/Data Connectors/ValenceSecurity_PUSH_CCP/ValenceSecurity_connectorDefinition.json Adds CCF connector definition (Customizable) with UI config, queries, and instructions.
Solutions/Valence Security/Data Connectors/ValenceSecurity_PUSH_CCP/ValenceSecurity_PollerConfig.json Adds Push dataConnector (poller config) targeting the DCR stream.
Solutions/Valence Security/Data Connectors/ValenceSecurity_PUSH_CCP/ValenceSecurity_DCR.json Adds DCR with two streams and transforms for TimeGenerated.
Solutions/Valence Security/Data Connectors/ValenceSecurity.json Deletes the deprecated legacy Data Collector API connector.
Solutions/Valence Security/Analytic Rules/ValenceAlerts.yaml Updates analytic rule to new connector ID, table, and field mappings; bumps rule version.
.script/tests/detectionTemplateSchemaValidation/ValidConnectorIds.json Registers ValenceSecurityCcfPush as a valid connector ID for validations.
.script/tests/KqlvalidationsTests/CustomTables/ValenceAlerts_CL.json Adds schema to support repo KQL validation tests for the new custom table.
Comments suppressed due to low confidence (1)

Solutions/Valence Security/Package/mainTemplate.json:1

  • Several ARM/packaging expressions appear malformed (missing closing ]] / ]), which will cause template parsing/deployment failures. Please fix the bracket/quote balancing for these [[...]] expressions (e.g., newGuid, concat(...), and the parameters(...) references under dcrConfig / auth) so they are syntactically valid for the packaging tool and downstream deployment.
{

Comment thread Solutions/Valence Security/Package/mainTemplate.json
Comment thread Solutions/Valence Security/Package/mainTemplate.json
Comment thread Solutions/Valence Security/Package/mainTemplate.json
Comment thread Solutions/Valence Security/Package/mainTemplate.json
Comment thread Solutions/Valence Security/Package/mainTemplate.json
Comment thread Solutions/Valence Security/Package/createUiDefinition.json
@v-maheshbh

Copy link
Copy Markdown
Contributor

Hi @shlomimatichin

Kindly review the Copilot comments added above and resolve them, if applicable and update the release notes with the latest version changes.

image

ensure that the existing ZIP file is not deleted during repackaging.

Thanks!

@shlomimatichin

Copy link
Copy Markdown
Contributor Author

done, its the same resolved comments from the previous PR (closed due to CI bug, nothing really changed)

@v-maheshbh

Copy link
Copy Markdown
Contributor

Hi @shlomimatichin

Kindly review above comments.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants