Skip to content

mathijsvermaat/Sentinel-Maturity

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

90 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Sentinel Maturity Model — Data Connector Guidance

A structured approach to Microsoft Sentinel data connector onboarding, retention, and detection rationale for the Dutch Security TS team.

Tip

This maturity model is the what-to-ingest layer. It is paired with the Guidance section that explains why each decision is made (risk, retention, cost, compliance), and the interactive Assessment Checklist that tracks how far an organisation has implemented it. Use the model to define the target, the guidance to justify it, and the assessment to measure progress.


Contents


Purpose

This guide provides a tiered maturity model for Microsoft Sentinel data connectors. It helps teams answer three questions:

  1. What data should we ingest? — Which connectors and tables are essential?
  2. Why should we keep it? — What is the security rationale (forensic readiness, detection, compliance)?
  3. How long should we retain it? — What retention periods align with regulatory and operational needs?

Guidance

Before diving into specific connectors and tables, review the strategic guidance that underpins the decisions in this maturity model:

Topic Description
Risk Considerations Why there is no one-size-fits-all logging configuration and how to apply a risk-based approach
Input/Output Strategy Gartner's SIEM input/output strategy — tiering telemetry for cost-effective security operations
Forensic Readiness Designing logging and retention for incident investigation from day one
Layered Detection Approach Why EDR alone is not sufficient and how SIEM-based logging provides defence in depth
Frameworks and Compliance MCSB, SFI, NIS2, and other regulatory standards that inform logging decisions
Budget and Cost Planning SOC budgeting and Microsoft Sentinel cost optimisation strategies
Retention Industry best practices for log retention — MCSB, NIST, CIS, NIS2, and GDPR mapped to Sentinel storage tiers

Procedures

Step-by-step guides for the operational tools used alongside this maturity model:

Procedure Tool Type Description
Workspace Usage Report Workbook Check free data connectors, ingestion benefit coverage, connector volumes, and retention settings
XDR Ingestion Calculator Script Estimate Defender XDR ingestion volumes before enabling the Sentinel connector
XDR Data Volume Insights KQL Query Measure Defender XDR and Entra ID table sizes, daily averages, and event counts to inform Analytics vs Data Lake tier decisions
Defender AMA Coverage Workbook Validate AMA deployment coverage and identify gaps in security event and syslog collection
Retention Insights Workbook Review table-level retention and archiving settings, evaluate Data Lake candidates, and estimate cost impact of plan changes

Tier Model

Tier Description When to adopt
Tier 1 Foundational — Essential connectors that every Sentinel deployment should have. Covers identity, endpoint, email, cloud activity, and server logs. Start here — the foundation for every organisation
Tier 2 Extended visibility — Network security, cloud posture, data protection, multi-cloud, endpoint compliance, and threat intelligence. Once Tier 1 is operational and the team is ready to broaden coverage
Tier 3 Advanced / Specialised — Full-spectrum monitoring including OT/IoT, CI/CD, SAP, databases, custom applications, and specialised integrations. When Tier 1 & 2 are in place and specialised workloads require visibility

Tier 1 Connectors (Foundational — Essential Connectors)

Connector Key Tables Licensing Benefit Free Ingestion
Microsoft Defender XDR DeviceEvents, AlertInfo, EmailEvents, IdentityLogonEvents, CloudAppEvents, ... M365 E5 / E5 Security Yes — ingestion to Analytics tier only via Microsoft Sentinel benefit for M365 E5 customers
Microsoft Entra ID SigninLogs, AuditLogs, AADNonInteractiveUserSignInLogs, AADRiskyUsers, AADRiskyServicePrincipals, ... Free data source (with conditional ingestion benefits in some E5 entitlement scenarios) Yes — free data connectors
Office 365 OfficeActivity M365 E3/E5 Yes — free data connector
Azure Activity Logs AzureActivity Any Azure subscription Yes — free data connector
Windows Security Events SecurityEvent / WindowsEvent Defender for Servers P2 Pooled 500 MB/day × Defender for Servers P2-licensed servers via Defender for Servers P2 (SecurityEvent stream only)
Syslog for Linux Syslog None None
Sentinel Health & Audit Diagnostics SentinelHealth, SentinelAudit Any Sentinel workspace Yes — SentinelHealth is not billable

Tier 2 Connectors (Extended Visibility)

Tier 2 extends monitoring into network security, cloud posture, data protection, multi-cloud, endpoint compliance, and threat intelligence. These connectors are aligned with frameworks like MCSB, NIST, CIS and more. Connectors marked conditional only apply when the relevant product or cloud is in use. Tier 2 is aligned with the ASD ACSC Best Practices for Event Logging and Threat Detection logging priorities.

Cloud Security Posture

Connector Key Tables Free Ingestion
Microsoft Defender for Cloud SecurityAlert, SecurityRecommendation Yes — SecurityAlert is free

Data Protection & Governance

Connector Key Tables Free Ingestion
Azure Key Vault AKVAuditLogs No
Microsoft Copilot / AI Governance OfficeActivity (Copilot), AzureDiagnostics (OpenAI) No
Microsoft Purview (Data Map / Discovery) PurviewDataSensitivityLogs No
Microsoft Purview Information Protection (Preview) MicrosoftPurviewInformationProtection No

Detection Enrichment

Connector Key Tables Free Ingestion
Threat Intelligence Platforms ThreatIntelIndicators, ThreatIntelObjects Yes — free data source

Endpoint Compliance

Connector Key Tables Free Ingestion
Microsoft Intune (Endpoint Management) IntuneAuditLogs, IntuneOperationalLogs, IntuneDevices, IntuneDeviceComplianceOrg No

Identity & Access (Extended)

Connector Key Tables Free Ingestion
Third-Party Identity (Okta, CyberArk, Ping Identity, BeyondTrust) Vendor-specific tables via API or CEF/Syslog No — conditional

Multi-Cloud

Connector Key Tables Free Ingestion
Amazon Web Services (AWS) AWSCloudTrail, AWSGuardDuty, AWSVPCFlow No
Google Cloud Platform (GCP) GCPAuditLogs No

Network Visibility

Connector Key Tables Free Ingestion
Azure Firewall AZFWNetworkRule, AZFWApplicationRule, AZFWDnsQuery, AZFWThreatIntel No
Azure WAF (Application Gateway / Front Door) ApplicationGatewayFirewallLog, FrontDoorWebApplicationFirewallLog No
DNS Security Logs DnsEvents, DnsInventory No
Microsoft Global Secure Access NetworkAccessTraffic No
VNet Flow Logs & Traffic Analytics NTANetAnalytics, NTAIpDetails No
Third-Party Network & Proxy Appliances (CEF/Syslog) CommonSecurityLog No — conditional

Tier 3 Connectors (Advanced / Specialised)

Tier 3 provides full-spectrum monitoring for mature organisations that have completed Tier 1 and Tier 2. These connectors cover OT/IoT, DevOps supply chain, databases, custom business applications, and advanced infrastructure telemetry.

Application & Workload Security

Connector Key Tables Free Ingestion
IIS / Web Server Logs W3CIISLog No
Microsoft Defender for Cloud Apps (Standalone) McasShadowItReporting No
SAP SAPAuditLog, ABAPAuditLog, SAPChangeDocuments No — separately licensed
SQL / Database Audit Logs SQLSecurityAuditEvents, CDBDataPlaneRequests No
Third-Party Applications (ServiceNow, Salesforce, Workday) Vendor-specific tables via API or CEF/Syslog No — conditional

Collaboration & Communication

Connector Key Tables Free Ingestion
Third-Party Collaboration (Slack, Zoom, Cisco Webex) Vendor-specific tables via API or webhook No — conditional

Custom Applications (Crown Jewels)

Connector Key Tables Free Ingestion
Custom Applications {AppName}_CL (custom tables) No

DevOps & CI/CD Security

Connector Key Tables Free Ingestion
Azure DevOps AzureDevOpsAuditing No
GitHub Enterprise GitHubAuditLogsV2_CL No
Third-Party DevOps (GitLab, Jenkins, Bitbucket) Vendor-specific tables via API or webhook No — conditional

Infrastructure & Platform

Connector Key Tables Free Ingestion
Azure Kubernetes Service (AKS) Audit AKSAudit, AKSAuditAdmin No
Azure Storage Account AzureMetrics, StorageBlobLogs, StorageQueueLogs, StorageTableLogs, StorageFileLogs No
Windows Forwarded Events (Advanced) WindowsEvent (PowerShell, Sysmon, AppLocker) No

OT / IoT Security

Connector Key Tables Free Ingestion
Microsoft Defender for IoT SecurityAlert (IoT) Yes — SecurityAlert is free
Third-Party OT / IoT (Claroty, Nozomi Networks, Armis) Vendor-specific tables via CEF/Syslog No — conditional

Retention Philosophy

For the full retention framework analysis including specific requirements from MCSB, NIST, CIS, NIS2, and GDPR, see Retention. For investigation readiness considerations, see Forensic Readiness. Our retention recommendations are informed by:

  • Microsoft Cloud Security Benchmark (MCSB) — Specifically controls LT-1 through LT-6 and IR-4/IR-5
  • Forensic readiness — The ability to investigate incidents that may have started weeks or months before detection (average dwell time in 2024: ~10 days for ransomware, but APTs can persist for months)
  • Layered security approach — Defence in depth requires correlated data across identity, endpoint, network, and cloud layers
  • Regulatory and compliance requirements — GDPR, NIS2, SOC 2, ISO 27001

Recommended Retention Tiers

Tier Duration Purpose Sentinel Feature
Analytics 90 days Active detection, hunting, and incident response Analytics logs (interactive)
Long-term 365 days Extended investigations, historical correlation, threat hunting Sentinel Data Lake (Lake)

Note

The default long-term retention for all Tier 1 tables is 365 days in the Sentinel Data Lake. Adjust per table based on compliance or forensic requirements.


Why a Layered Approach?

For the full rationale, see Layered Detection Approach. EDR solutions like Microsoft Defender for Endpoint are essential but not sufficient on their own. A layered approach combining EDR with native OS logging (Windows Security Events, Syslog) provides defence in depth:

  • EDR can be bypassed — attackers continuously develop techniques to evade endpoint detection. Native OS logs provide an independent audit trail that persists even if EDR is tampered with.
  • Forensic readiness — native logs provide authoritative evidence admissible in investigations, complementing EDR telemetry.
  • Correlation across data sources — combining identity, endpoint, email, and cloud activity logs enables detection of multi-stage attacks that no single source can catch alone.

Recommended Reading

Title Description Link
The Evolution of EDR Bypasses Historical timeline showing how EDR bypass techniques evolve, reinforcing why native logs are essential as a fallback CovertSwarm
MDE Telemetry Unreliability and Log Augmentation In-depth analysis of MDE telemetry gaps, capping behaviour, and why native Windows logs are essential for complete forensic coverage FalconForce
Cloud Forensics: Forensic Readiness and IR in Azure Virtual Desktop Demonstrates a layered approach combining EDR and native logging for incident response in cloud environments Microsoft Community Hub
Windows Event Log Analysis: Techniques for Every SOC Analyst Practical guide on using Windows Security Events for detection, showing their value alongside EDR CyberDefenders Blog
Sentinel Data Connectors: What Actually Matters Practical guidance on which Sentinel data connectors to prioritize IT Professor

Tools

To help identify retention settings, monitor ingestion volumes, estimate costs, and validate data connector coverage. Each tool has a step-by-step walkthrough in the Procedures section.

Tool Type Purpose Source Guide
Workspace Usage Report Workbook Monitor ingestion volumes per table, identify cost optimisation opportunities, and validate data connector health across all connectors Sentinel Content Hub (search "Workspace Usage") Walkthrough
Defender AMA Coverage Workbook Validate Defender for Endpoint and AMA agent deployment coverage, identify gaps in security event and syslog collection GitHub — mathijsvermaat/Defender-AMA-coverage Walkthrough · Blog
XDR tables to Sentinel ingestion calculator Script Estimate Defender XDR ingestion volumes from the Advanced Hunting API before enabling the Sentinel connector GitHub — mathijsvermaat/DefenderIngestToSentinel Walkthrough
XDR Data Volume Insights KQL Query Measure Defender XDR and Entra ID table sizes, daily averages, and event counts to inform Analytics vs Data Lake tier decisions Run in Advanced Hunting in Defender portal Walkthrough
Retention Insights Workbook Review table-level retention and archiving settings, evaluate Data Lake candidates, and estimate cost impact of plan changes Github — Azure-Sentinel/Workbooks Walkthrough

Assessment Checklist

Use the interactive Sentinel Maturity Assessment Checklist to track progress during a connector onboarding engagement. The checklist covers every Tier 1 connector with per-table checks, retention validation, and configuration items. Each section has a comment field for rationale and notes.

The tool offers two modes, selectable from the header toggle:

  • Full — every Tier 1/2/3 connector with detailed per-item checks (Setup, Tables, Content Hub, Retention, Validation), per-table tier-change planning (Analytics ↔ Lake) and per-table retention planning, plus a comments field per connector. Use for delivery engagements, formal assessments, and customer hand-off.
  • Lite — a focused four-check landing page for quick conversations. Check 2 ("Defender XDR — Per-Table Ingestion") mirrors the same per-table tier and retention planner used in Full mode and feeds the same to-do output (signed GB/day delta + per-tier breakdown). The accepted-risk panel and to-do summary remain visible. Use for intro calls, executive briefings, and demos.

Features: save/load progress (JSON), export to PDF, export to Excel.


References


Reference Index

For a consolidated, categorised index of every external URL cited across this maturity model (Microsoft Learn docs, Azure pricing pages, tools and workbooks, blogs, standards and frameworks), see references.md.

Last updated: April 2026

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors