Auditing Ansible at Scale with Structured Action Logging (JSONL)

DevOpsAnsibleLoggingComplianceSecurityObservability

Why stdout Is Not an Audit Trail

Ansible is excellent at executing tasks but weak at explaining afterwards what actually happened in a machine-readable way.

When managing hundreds of hosts across multiple environments, tracking what changed, when, and why becomes critical for:

  • Security audits and compliance
  • Change management and rollback planning
  • Performance analysis and optimization
  • Incident investigation

This article introduces rf_action_logger, an Ansible meta-logger that emits structured JSONL execution events for every action.

The Problem with Traditional Ansible Logging

Standard Ansible output is designed for human readability:

TASK [Ensure SSH config is hardened] **************
changed: [srv-wireguard-01]
ok: [srv-database-01]
failed: [srv-legacy-02] => {"msg": "Permission denied"}

This format is:

  • Hard to parse programmatically
  • Lacks structured metadata
  • Difficult to correlate across runs
  • Impossible to query efficiently

Execution as Data

rf_action_logger transforms every task execution into a self-contained JSON record:

{
  "timestamp": "2026-01-18T04:12:31.492Z",
  "task": "Ensure SSH config is hardened",
  "action": "ansible.builtin.template",
  "host": "srv-wireguard-01",
  "changed": true,
  "failed": false,
  "duration_ms": 127,
  "customer": "c_00000_acme",
  "environment": "prod",
  "playbook": "security-baseline.yml",
  "run_id": "550e8400-e29b-41d4-a716-446655440000"
}

Why JSONL

JSON Lines format offers several advantages:

  • Append-only: New events simply append to the file
  • Stream-friendly: Process logs line-by-line without loading everything
  • Tool compatibility: Works with jq, grep, and standard Unix tools
  • SIEM-ready: Direct ingestion into Elasticsearch, Splunk, etc.
  • Diffable: Compare runs to see exactly what changed

Real-World Use Cases

1. Compliance Reporting

# Find all changes in production this month
jq 'select(.environment == "prod" and .changed == true)' \
  /var/log/ansible/2026-01-*.jsonl

2. Failure Analysis

# Track failure patterns across hosts
jq 'select(.failed == true) | 
  {task: .task, host: .host, error: .msg}' \
  actions.jsonl | sort | uniq -c

3. Performance Monitoring

# Find slow tasks
jq 'select(.duration_ms > 5000) | 
  {task: .task, duration: .duration_ms}' \
  actions.jsonl | sort -k2 -nr

4. Change Tracking

# Generate change report for specific host
jq --arg host "srv-database-01" \
  'select(.host == $host and .changed == true)' \
  actions.jsonl

Configuration

Enable logging with environment variables:

export RF_ACTION_LOGGER_ENABLED=true
export RF_ACTION_LOGGER_PATH=/var/log/ansible/actions.jsonl
export RF_ACTION_LOGGER_ROTATE=daily
export RF_ACTION_LOGGER_RETAIN_DAYS=90

No playbook changes required — the logger hooks into Ansible's callback system.

Advanced Features

Structured Context

Each log entry includes execution context:

{
  "git_commit": "a3f4c2d",
  "git_branch": "main",
  "operator": "patrick@mpowr-it.com",
  "ci_pipeline": "jenkins-123"
}

Sensitive Data Handling

# Mark sensitive results
- name: Generate API token
  command: generate-token.sh
  no_log: true  # rf_action_logger respects this

Custom Fields

# Add metadata to specific tasks
- name: Database migration
  command: migrate.sh
  vars:
    rf_action_logger_meta:
      criticality: high
      approval_ticket: "CHG-2026-0142"

Integration Examples

Elasticsearch Pipeline

# Stream to Elasticsearch
tail -f actions.jsonl | \
  jq -c '. + {[@timestamp]: .timestamp}' | \
  curl -X POST "elasticsearch:9200/_bulk" \
    --data-binary @-

Prometheus Metrics

# Export metrics
for line in open('actions.jsonl'):
    event = json.loads(line)
    if event['failed']:
        ansible_task_failures.labels(
            task=event['task'],
            host=event['host']
        ).inc()

Production Patterns

Log Rotation

/var/log/ansible/*.jsonl {
    daily
    rotate 90
    compress
    delaycompress
    notifempty
    create 0640 ansible ansible
}

Centralized Collection

# Ship logs to central store
- name: Upload execution logs
  ansible.builtin.synchronize:
    src: "{{ rf_action_logger_path }}"
    dest: "rsync://logstore/ansible/{{ inventory_hostname }}/"
    mode: push

Performance Considerations

rf_action_logger adds minimal overhead:

  • ~1-2ms per task
  • Async writes don't block execution
  • Configurable buffering for high-frequency tasks

For a typical playbook with 100 tasks across 50 hosts, this adds less than 1 second total.

Security Benefits

  1. Forensic Analysis: Reconstruct exact execution flow after incidents
  2. Change Attribution: Link every change to operator and approval
  3. Anomaly Detection: Spot unusual patterns in automation behavior
  4. Compliance Evidence: Prove controls were applied consistently

Conclusion

rf_action_logger turns Ansible executions into auditable data, enabling traceability, compliance, and post-run analysis at scale.

For teams running Ansible in regulated environments or at scale, structured logging transforms operations from black-box automation to transparent, auditable infrastructure management.

The plugin is open source and has been proven in environments with strict audit requirements and thousands of managed hosts.

rf_action_logger has been deployed across numerous high-security environments, from financial services to critical infrastructure providers.

Auditing Ansible at Scale with Structured Action Logging (JSONL) - Patrick Paechnatz