Skip to content

Guides & Downloads

Reference guides and downloads for Log Processor. These are versioned — the links below point at the current release (26h4c) published on perfware.cloud.

Guides

Downloads


LogProcessor Helper Scripts

26h4c 2026-11-10

Helper scripts for operating a deployed Log Processor stack — user and group management, SSO, domain setup, snapshots, diagnostics, cross-account forwarding, S3 access logging, and cleanup.

Prerequisites

  • Windows 10/11 or Windows Server 2016+ (for .cmd scripts)
  • Linux, macOS, or Git Bash on Windows (for .sh scripts)
  • AWS CLI v2.13+ (latest recommended)
  • AWS credentials configured (aws configure, SSO, or environment variables)
  • PowerShell 5.1+ (included with Windows 10/11)
  • Python 3.10+ (for check-quota.py and cleanup.py)

Run scripts from any directory with AWS CLI access to the target account and region. Region defaults to us-east-1 if not specified, except cleanup.py, which requires region explicitly. Stack name is case-sensitive and must match the CloudFormation stack name.

Most .cmd scripts have an equivalent .sh script with identical functionality. Exceptions: check-quota / cleanup are Python tools.

Contents


User Management

users.cmd / users.sh

Manage Cognito users for OpenSearch Dashboards and Subscription Editor access. Each user authenticates via the Cognito-backed ALB and can access Dashboards, the Subscription Editor, and any SSO-protected resources.

users.cmd <stack-name> <action> [args] [region]
Action Description
create <email> [temp-password] Create a user (forced password change on first login)
list List all users with username, email, status, and creation date
delete <email> Permanently remove a user
reset <email> [temp-password] Reset password (user sets new one on next login)
disable <email> Block a user from logging in (preserves account)
enable <email> Re-enable a previously disabled user

Examples

users.cmd LogProcessor create admin@company.com
users.cmd LogProcessor create admin@company.com "MyTemp!Pass99" us-west-2
users.cmd LogProcessor list
users.cmd LogProcessor disable former-employee@company.com
users.cmd LogProcessor reset admin@company.com

Notes

  • Default temporary password is auto-generated if not specified.
  • Federated (SSO) users appear with their provider prefix in Username.
  • Email column may show None for SSO users (email is in Username).

Requires: AWS CLI, Cognito UserPool deployed (Essential+ tier).


Group & Role Management

groups.cmd / groups.sh

Manage Cognito user groups for role-based access control (RBAC) in the Subscription Editor. Groups determine what each user can do.

groups.cmd <stack-name> <action> [args] [region]
Action Description
list-groups List all groups in the user pool
create-group <name> [description] Create a new group
delete-group <name> Remove a group
add-to-group <email> <group> Assign a user to a group
remove-from-group <email> <group> Remove a user from a group
list-users [group] List all users, or users in a specific group

Examples

groups.cmd LogProcessor create-group admin "Full editor access"
groups.cmd LogProcessor create-group viewer "Read-only access"
groups.cmd LogProcessor add-to-group admin@company.com admin
groups.cmd LogProcessor add-to-group analyst@company.com viewer
groups.cmd LogProcessor list-users admin
groups.cmd LogProcessor list-groups

Editor RBAC (Advanced+ tier with custos feature)

Membership Access
admin group Full access: Save, Sync, Rollback, Import
viewer group Read-only: can view but cannot save (server returns 403)
no group Blocked: Access Denied page
no groups configured (custos inactive) Everyone has full access

Notes

  • Users must log out and back in for group changes to take effect.
  • ALB sessions last 1 hour (fixed window, not sliding).
  • SSO users must also be assigned to groups after first login.

Requires: AWS CLI, Cognito UserPool deployed (Essential+ tier).


SSO / Identity Federation

sso.cmd / sso.sh

Manage external SSO (SAML/OIDC) identity providers on the Cognito User Pool. Enables federated login from corporate identity providers (Okta, Azure AD, Google Workspace, ADFS, Ping, etc.) to Dashboards and the Subscription Editor.

sso.cmd <stack-name> <action> [args]
Action Description
setup-saml <name> <metadata-url> Add a SAML provider
setup-oidc <name> <issuer> <client-id> <client-secret> Add an OIDC provider
list List configured providers
remove <name> Remove a provider
info Show URLs for IdP configuration

Examples

sso.cmd LogProcessor info
sso.cmd LogProcessor setup-saml Okta https://dev-123.okta.com/app/xxx/sso/saml/metadata
sso.cmd LogProcessor setup-oidc AzureAD https://login.microsoftonline.com/tenant/v2.0 client-id client-secret
sso.cmd LogProcessor list
sso.cmd LogProcessor remove Okta

Workflow

  1. Run info to get the ACS URL and Entity ID.
  2. Configure your IdP with those values.
  3. Run setup-saml or setup-oidc with the IdP metadata/config.
  4. Users see "Sign in with <name>" on the login page.
  5. After first SSO login, assign groups: groups.cmd <stack> add-to-group <email> admin.

Notes

  • Multiple providers can coexist (e.g. Okta + Azure AD).
  • Username/password login remains available alongside SSO.
  • Federated users appear as <Provider>_<email> in user listings.
  • Group assignment is required for editor RBAC (custos feature).

Requires: AWS CLI, Cognito UserPool deployed (Essential+ tier).


Domain & Certificate Setup

setup-domain.cmd / setup-domain.sh

Configure a custom domain (e.g. dashboards.company.com) for the OpenSearch Dashboards ALB. Handles ACM certificate creation, DNS validation, and Route 53 alias record creation.

setup-domain.cmd cert-only <domain> <hosted-zone-id> [region]
setup-domain.cmd <stack-name> <domain> <hosted-zone-id> [region]
Mode Description
cert-only Run before deployment to request and validate an ACM certificate. Add the certificate ARN to your config before deploying.
<stack> Run after deployment to create the Route 53 CNAME/alias to the ALB.

Examples

setup-domain.cmd cert-only dashboards.company.com Z0123456789ABC
setup-domain.cmd LogProcessor dashboards.company.com Z0123456789ABC

Workflow

  1. Run cert-only to get the certificate ARN.
  2. Add certificateArn and domain to your config.
  3. Deploy the stack.
  4. Run with stack name to create the DNS record.
  5. Access dashboards at https://dashboards.company.com.

Notes

  • Certificate validation uses DNS (Route 53 auto-validation).
  • Certificate must be in the same region as the ALB.
  • Propagation takes a few minutes after DNS record creation.

Requires: AWS CLI, Route 53 hosted zone for the domain.


OpenSearch Snapshots

snapshot.cmd / snapshot.sh

Manage OpenSearch manual snapshots for backup and recovery. Snapshots are stored in the S3 snapshot bucket configured during deployment.

snapshot.cmd <stack-name> <action> [name] [region]
Action Description
take <name> Create a manual snapshot of all indices
list List all available snapshots
status <name> Check progress/status of a snapshot
delete <name> Remove a snapshot from the repository

Examples

snapshot.cmd LogProcessor take pre-upgrade-backup
snapshot.cmd LogProcessor list
snapshot.cmd LogProcessor status pre-upgrade-backup
snapshot.cmd LogProcessor delete old-snapshot

Notes

  • Snapshots are incremental (only changed data is stored).
  • The snapshot repository is automatically registered during deployment.
  • Restore via OpenSearch Dev Tools: POST /_snapshot/repo/name/_restore.
  • Lifecycle policies auto-expire old snapshots per config.

Requires: AWS CLI, OpenSearch snapshot repository configured (Essential+ tier).


Diagnostics & Support

support-bundle.cmd / support-bundle.sh

Collect a comprehensive diagnostic bundle for troubleshooting. Gathers configuration, logs, metrics, and health status from all stack components.

support-bundle.cmd <stack-name> [hours] [region]
Argument Description
stack-name CloudFormation stack name
hours Hours of log/metric history to collect (default: 24)
region AWS region (default: us-east-1)

Collects

  • CloudFormation stack status and parameters
  • Lambda configuration, recent logs, errors, and environment variables
  • SQS queue depths and dead letter queue messages
  • OpenSearch domain health, cluster status, and index stats
  • CloudWatch metrics and alarm states
  • SNS topic subscriptions
  • S3 bucket status and object counts
  • Firehose delivery stream configuration
  • Editor Lambda logs
  • ALB health and target group status
  • Nginx access/error logs
  • Current subscriptions.json
  • Service quotas and limits
  • Cognito configuration
  • Tier and build information

Output: support-bundle/<stack-name>-<timestamp>/ folder with JSON/text files. Zip the folder and email to support@perfware.cloud.

Examples

support-bundle.cmd LogProcessor
support-bundle.cmd LogProcessor 48 us-west-2

Requires: AWS CLI, read access to stack resources.

check-quota.cmd / check-quota.sh / check-quota.py

Pre-deployment check of AWS service quotas against Log Processor tier requirements. Identifies potential deployment blockers before you start.

check-quota.cmd <tier> [region]
check-quota.sh  <tier> [region]
Argument Description
tier Target tier: basic, essential, advanced, enterprise
region AWS region (default: us-east-1)

Checks

  • VPC count and available headroom
  • Lambda concurrent executions
  • S3 bucket count
  • KMS key count
  • DynamoDB table count
  • Firehose delivery streams
  • OpenSearch domain count and instance limits
  • ALB count
  • Cognito user pool count
  • SNS topic count
  • VPC interface endpoint count

Output: PASS/WARN/FAIL per quota with current usage, limit, and headroom. Warns at 80% utilization. Fails if deployment would exceed limits.

Examples

check-quota.cmd enterprise
check-quota.cmd basic us-west-2

Requires: Python 3.10+, AWS CLI, service-quotas and resource read permissions.


Cross-Account Log Forwarding

get-destination.cmd / get-destination.sh

Alternative to using the Editor.

Display the CloudWatch Logs destination ARN for a Log Processor stack. This ARN is needed by external accounts to set up log forwarding.

get-destination.cmd <stack-name> [region]

Example

get-destination.cmd LogProcessor

Output

arn:aws:logs:us-east-1:000000000000:destination:LogProcessor-destination

Notes

  • Share this ARN with external account administrators.
  • The destination accepts logs from accounts listed in CrossAccountIds or any account in the organization specified by OrganizationId.

Requires: AWS CLI, logs:DescribeDestinations permission.

setup-cross-account.cmd / setup-cross-account.sh

Alternative to using the Editor.

Manage cross-account log subscriptions. Run in the external (source) account to forward CloudWatch logs to a central Log Processor destination.

setup-cross-account.cmd <action> [args]
Action Description
setup <log-group> <destination-arn> [filter] [region] Create subscription filter
list <stack-name> <log-group> [region] List filters for a stack on a log group
remove <stack-name> <log-group> [region] Remove a subscription filter

Examples

setup-cross-account.cmd setup /aws/lambda/my-fn arn:aws:logs:us-east-1:000000000000:destination:log-dest-LogProcessor-000000000000-us-east-1
setup-cross-account.cmd setup /aws/lambda/my-fn arn:aws:logs:... "ERROR" us-east-1
setup-cross-account.cmd list LogProcessor /aws/lambda/my-fn
setup-cross-account.cmd remove LogProcessor /aws/lambda/my-fn

Workflow

  1. In the Log Processor account: deploy with crossAccountIds including this account.
  2. In the Log Processor account: run get-destination.cmd to get the ARN.
  3. In this (external) account: run setup-cross-account.cmd setup ....
  4. In the Log Processor account: add a subscription entry with accountId matching.

Prerequisites (in the Log Processor account)

  • Enterprise tier deployed with crossAccountEnabled: true.
  • This account's ID in the CrossAccountIds parameter, or the OrganizationId parameter set to the shared AWS Organization ID.

Notes

  • Filter pattern is optional (empty = all log events).
  • Each log group can have max 2 subscription filters (CloudWatch limit).
  • list and remove require the stack name to identify the correct filter (supports multiple Log Processor stacks with different names).
  • Use list to verify filters are active.

Requires: AWS CLI, logs:PutSubscriptionFilter, logs:DeleteSubscriptionFilter, logs:DescribeSubscriptionFilters.

setup-remote-access.cmd / setup-remote-access.sh

Set up IAM roles in a remote (source) account to allow the central Log Processor to browse log groups, manage subscription filters, and receive logs via Firehose.

setup-remote-access.cmd setup <central-account-id> [region] [--org <org-id>]
setup-remote-access.cmd remove [region]
setup-remote-access.cmd status [region]
Action Description
setup <central-account-id> [region] [--org <org-id>] Create IAM roles trusting the central account
remove [region] Delete both IAM roles
status [region] Show current role configuration

Examples

setup-remote-access.cmd setup 000000000000
setup-remote-access.cmd setup 000000000000 us-east-1 --org o-abc123
setup-remote-access.cmd status
setup-remote-access.cmd remove

Roles created

Role Purpose
LogProcessor-remote-access Assumed by central Lambda to manage subscription filters
LogProcessor-logs-delivery Assumed by CloudWatch Logs to deliver to central Firehose

Workflow

  1. In the central account: deploy Enterprise tier with crossAccountEnabled.
  2. In this (remote) account: run setup-remote-access.cmd setup <central-account-id>.
  3. The central Log Processor can now browse and manage filters in this account.
  4. No further configuration needed; the editor auto-discovers this account.

Options (setup only)

  • --org <org-id> — also trust any account in the specified AWS Organization.

Notes

  • Run this in the remote account (not the central Log Processor account).
  • The setup command is idempotent (safe to re-run to update trust policies).
  • Use status to verify current role configuration.
  • Use remove to cleanly delete both roles and their policies.

Requires: AWS CLI, IAM permissions (iam:CreateRole, iam:PutRolePolicy, iam:DeleteRole, iam:DeleteRolePolicy), sts:GetCallerIdentity.

add-account.cmd / add-account.sh

Alternative to using the Editor.

Register or remove remote account IDs in the central Log Processor so they appear in the Subscription Editor's account dropdown immediately, without waiting for log events to arrive.

add-account.cmd <stack-name> <account-id> [region]
add-account.cmd <stack-name> list [region]
add-account.cmd <stack-name> remove <account-id> [region]
Action Description
<account-id> [region] Add a 12-digit account ID
list [region] Show currently registered accounts
remove <account-id> [region] Remove an account from the list

Examples

add-account.cmd LogProcessor 111222333444
add-account.cmd LogProcessor 111222333444 us-west-2
add-account.cmd LogProcessor list
add-account.cmd LogProcessor remove 111222333444

Workflow

  1. In the remote account: run setup-remote-access to create IAM roles.
  2. In the central account: run add-account to register the account ID.
  3. The account immediately appears in the Subscription Editor dropdown.

Notes

  • Run this in the central account (where Log Processor is deployed).
  • Account IDs must be exactly 12 digits.
  • Adding an already-registered account is a no-op (exits cleanly).
  • Accounts are stored in config/accounts.json in the assets bucket.

Requires: AWS CLI, SSM read access, S3 read/write access to assets bucket, Python 3.10+.


S3 Access Logging

add-access-logging.cmd / add-access-logging.sh

Enable S3 server access logging on any bucket, routing logs to the stack's access log bucket. Logs are automatically partitioned and queryable via Athena.

add-access-logging.cmd <stack-name> <action> [args] [region]
Action Description
list Show buckets that have delivered logs
add <source-bucket> Enable access logging on a bucket
remove <source-bucket> Disable access logging (existing logs retained)

Examples

add-access-logging.cmd LogProcessor add my-data-bucket
add-access-logging.cmd LogProcessor add my-other-bucket
add-access-logging.cmd LogProcessor list
add-access-logging.cmd LogProcessor remove my-data-bucket

Query logs in Athena

SELECT * FROM access_logs
WHERE bucket = 'my-data-bucket' AND date = '2026/05/03'

Notes

  • Logs may take up to 1 hour to appear after enabling.
  • Works with any source bucket encryption (SSE-S3, SSE-KMS, CMK).
  • No additional IAM permissions needed for same-account buckets.
  • Logs use date-partitioned prefix format for efficient Athena queries.

Requires: AWS CLI, S3 permissions on both source and access log buckets.

setup-replication.cmd / setup-replication.sh

Cross-account S3 access log replication. Run in the external account to replicate access logs from your buckets to a central Log Processor stack's access log bucket for unified Athena querying.

setup-replication.cmd <action> [args]
Action Description
setup <target-bucket> [region] [options] One-time setup: create log bucket, role, rule
add <source-bucket> [region] Enable logging on a source bucket
verify <source-bucket> [region] Check logging and replication status
remove [region] Remove replication config

Options (setup only)

  • --expire-days N — delete objects from local log bucket after N days.
  • --noncurrent-days N — delete noncurrent versions after N days.

Examples

setup-replication.cmd setup logprocessor-access-logs-000000000000-us-east-1
setup-replication.cmd add my-production-bucket
setup-replication.cmd add my-staging-bucket
setup-replication.cmd verify my-production-bucket
setup-replication.cmd remove

Workflow

  1. In the Log Processor account: deploy with crossAccountIds including this account, or set OrganizationId to allow all accounts in the organization.
  2. In this account: run setup (creates local log bucket + replication to target).
  3. In this account: run add for each bucket you want to monitor.
  4. Logs replicate to the central bucket and are queryable via Athena.

Query in Athena (Log Processor account)

SELECT * FROM access_logs
WHERE bucket = 'my-production-bucket' AND date = '2026/05/04'

Prerequisites (in the Log Processor account)

  • Enterprise tier deployed.
  • This account's ID in the CrossAccountIds parameter, or the OrganizationId parameter set to the shared AWS Organization ID.

Requires: AWS CLI, IAM permissions (iam:CreateRole, iam:PutRolePolicy), S3 permissions on source and local log buckets.


Cleanup

cleanup.py

Clean up orphaned resources after stack deletion. CloudFormation may leave behind resources with RETAIN policies or resources created at runtime.

python cleanup.py <stack-name> <region> [-force|-confirm]
Argument Description
stack-name CloudFormation stack name (case-sensitive)
region AWS region (required)
Mode Behavior
(default) Dry run — shows what would be deleted without acting
-confirm Interactive — prompts for each resource before deleting
-force Deletes all matched resources without prompting

Cleans up

  • S3 buckets (empties and deletes, including versioned objects)
  • Athena workgroup (<stack-name>-workgroup)
  • CloudWatch log groups (Lambda, custom resources)
  • CloudWatch Logs resource policies
  • Lambda VPC ENIs (force-detach from VPC)
  • OpenSearch domains
  • Orphan Lambda functions
  • KMS keys (schedules deletion, 7-day window)
  • Orphan VPCs and their resources (endpoints, subnets, SGs, route tables)

Examples

python cleanup.py LogProcessor us-east-1           # dry run
python cleanup.py LogProcessor us-east-1 -confirm  # interactive
python cleanup.py LogProcessor us-east-1 -force    # delete all

Notes

  • Always run dry mode first to review what will be deleted.
  • Uses safe matching: regex word-boundary for S3, exact prefix for Lambdas, exact tag match for KMS (prevents accidental cross-stack deletion).
  • Prints a Resource Explorer verification hint at the end.

Warning: With -force, your S3 data (logs, datalake, snapshots) is permanently deleted. To preserve data, copy it first with aws s3 sync.

Requires: Python 3.10+, boto3, appropriate IAM permissions for each resource type.


Defaults & Notes

  • Region defaults to us-east-1 if not specified, except cleanup.py, which requires region as an explicit argument.
  • Stack name is case-sensitive and must match the CloudFormation stack name.
  • All scripts exit with code 0 on success, non-zero on error.
  • .sh scripts require chmod +x <script>.sh before first use.
  • Scripts auto-detect resources via CloudFormation stack outputs and tags.