Skip to content

Pattern Detection

Log Processor includes 50+ context-aware regex pattern detectors that scan log messages inline during processing, at no additional cost. There are no external API calls and no per-scan fees — no data leaves your VPC.

Pattern detection

Built-in detectors

Built-in patterns cover:

  • Identity — SSN, passport, driver's license, date of birth.
  • Financial — credit cards (with Visa/Mastercard validation), IBAN.
  • Contact — email, phone, IP address.
  • Credentials — AWS access keys, secret keys.
  • SQL — SELECT, INSERT, DROP, and injection attempts.

Each pattern is categorized by type (PII, PHI, Financial / FIN, Secret, SQL, APP) and by severity (High, Medium, Low).

Three modes

Configure each pattern per stream to:

  • Tag — annotate the event for downstream alerting.
  • Redact — mask the sensitive value.
  • Filter — drop the matching event entirely.

Custom patterns

Add your own custom patterns with a category and severity directly in the Subscription Editor — no code changes or redeployment needed. Pattern names must be alphanumeric with underscores.

Querying detections

Detected patterns are indexed as structured fields and are queryable in both OpenSearch and Athena, so you can search for specific pattern types across your entire log history.

Enhanced pattern metrics (Advanced tier and above)

On the Advanced tier and above, each detection is emitted as a CloudWatch metric with PatternName, Category, and Severity dimensions. This enables multi-dimensional querying — graph individual patterns over time, aggregate by severity, or slice by category — plus targeted dashboards and threshold alarms. Aggregate pattern metrics (total detections and filtered counts) are available on all tiers at no additional cost.

Anomaly detection

On the Advanced tier and above, create a CloudWatch alarm on any pattern-metric dimension (for example, "alert when SSN detections exceed 10 in 5 minutes" or "alert on any High severity detection") and route it to the stack's SNS topic. In CloudWatch, go to Alarms → Create Alarm, select the stack's metric namespace, filter by PatternName, Category, or Severity, and set your threshold.

AI Monitor correlation

Pattern rules include an AI Monitor Correlate flag that marks a pattern as eligible for cross-log correlation analysis by the companion AI Monitor product. Integration with AI Monitor is seamless and can eliminate the need for manual alarms; custom integration is available as a professional-services engagement.

Why regex

Regex pattern detection offers speed, precision, and immediate deployment without resource-heavy infrastructure. When you know your system's log schemas, regex lets you hardcode known patterns, filter predictable noise, and alert reliably without training or maintaining models.