Reversible OS Hardening: Profile Stacks, Snapshots and Resets

The One-Way Problem

Most hardening automation is a one-way street. It writes sysctl files, adds kernel arguments, remounts filesystems with noexec, disables services and removes packages. If a setting breaks an application, the way back is manual: find out what the value was before, on that specific host, and hope nobody changed it in the meantime.

That is tolerable for one baseline on uniform hosts. It stops being tolerable when hosts carry several baselines at once: a national baseline such as BSI IT-Grundschutz, a vendor or protection-profile guideline such as OSPP on Enterprise Linux, plus an environment-specific profile for directory-integrated hosts.

This article describes a profile engine that makes those stacks explicit and makes individual profiles reversible.

Profiles as Data

A profile is not a role. It is a named set of values inside each hardening category:

hardening:
  security:
    kernel:
      sysctl:
        baseline:
          kernel.kptr_restrict: 2
          kernel.yama.ptrace_scope: 2
          net.ipv4.ip_forward: 0
        ospp:
          kernel.kptr_restrict: 1
      modules_blacklist:
        baseline: [cramfs, freevxfs, hfs, hfsplus, jffs2, squashfs, udf, usb-storage]
        ospp: [bluetooth, atm, can]
    boot:
      cmdline:
        baseline: [audit=1, kaslr]
        ospp: [pti=on, vsyscall=none, page_poison=1, slub_debug=P]

A host declares its stack:

# host_vars/c_00000_acme/prod/101---app-01.yml
_active_hardening_profiles: [baseline, ospp]

The roles that apply settings never look at profile names. They consume computed facts: _kernel_sysctl_effective, _boot_cmdline_effective, _ssh_effective, _pam_effective, _mounts_effective. Adding a profile is a data change, not a code change.

Two Kinds of Merge

The categories do not all merge the same way, and getting this right is most of the design.

Keyed categories: the last profile wins. sysctl, SSH, PAM and mount options are key–value settings. A later profile replaces the value of an earlier one for the same key; keys only one profile sets are kept.

Additive categories: union. Module blacklists, disabled services, kernel arguments and audit rule files are sets. Every profile in the stack adds to them, and none can remove what another added.

The distinction follows from what the settings mean. Two profiles disagreeing on kernel.kptr_restrict need a winner. Two profiles each blacklisting modules simply want both lists applied.

Guard Rails on the Stack

The engine normalizes and validates the stack before computing anything:

• baseline is always first. It is prepended if missing, so no host can end up without the base layer.
• Unknown names fail the run. A profile name must be defined in at least one category. Without this check, a typo (osp instead of ospp) would be ignored and the host deployed without the profile, green.
• Order constraints are enforced. A directory-integration profile re-enables GSSAPI authentication that OSPP disables. Stacked the wrong way round, OSPP wins and every Kerberos login fails after deployment. The engine asserts the order instead of leaving it to documentation.

Overrides Are Recorded, Not Hidden

"Last wins" can loosen a setting. In the example above, OSPP sets kernel.kptr_restrict to 1 where the baseline wanted 2, and lowers the minimum password length from 14 to 12.

That can be entirely intentional: the host is assessed against OSPP, and OSPP’s value is the one that applies. But it must be visible. The engine computes an override map while merging:

_kernel_sysctl_overrides:
  kernel.kptr_restrict:
    from_profile: baseline
    from_value: 2
    to_profile: ospp
    to_value: 1

The map is written to a metadata file on the host (/etc/hardening/hardening-profiles.yaml) together with the active stack. Verification tooling reports these controls as overridden, not failed (see Verifying Hardening Against Several Catalogues at Once).

Snapshots: Capture Before You Change

Reversibility needs one thing a playbook usually does not have: the state of the host before the profile was applied.

The engine captures it at a precise moment: on the first deployment in which a profile is active, before any hardening role runs. For every item the profile touches, it reads the current value:

Category	Read with
sysctl	sysctl -e <keys>
kernel arguments	grubby --info=DEFAULT
mount options	findmnt -n -o OPTIONS <mountpoint>
services	systemctl is-enabled <unit>
packages	rpm -q <packages>

The result is written once:

# /etc/hardening/snapshots/ospp-baseline.yml  (root, 0600)
# Written once, before the profile was applied; reset mode restores from this file.
profile: ospp
stack_before: [baseline]
captured_by_release: 2.4.0
sysctl:
  kernel.kptr_restrict: '0'
cmdline:
  pti=on: false
  vsyscall=none: false
mounts:
  /dev/shm: rw,nosuid,nodev
services:
  bluetooth.service: enabled
packages:
  telnet: absent
captured_at: '2026-01-13T09:12:44Z'

Two rules make snapshots trustworthy:

• Never overwritten. The file is written with force: false. A later deployment, after the profile has changed the host, can never replace the original state with a hardened one.
• No snapshot, no reset. A profile that was already active before snapshot support existed has no recorded pre-change state. The engine says so and refuses to reset it, instead of guessing.

Mount options get one extra filter: findmnt also reports kernel-supplied values such as seclabel or size=, which are not valid in /etc/fstab. Only fstab-valid flags are stored.

Resetting a Profile

A reset is declared, like everything else:

_active_hardening_profiles: [baseline, ospp]
_reset_hardening_profiles:  [ospp]

The reset profile leaves the stack, so every keyed setting falls back to the remaining profiles automatically. Items that only the reset profile set are restored from its snapshot:

• sysctl values are written back.
• mount options are reapplied.
• kernel arguments the profile added are removed, if they were absent before.
• services get their previous enabled or disabled state.
• packages the profile removed are reinstalled, if they were installed before.

Audit rule files and module blacklist entries need no snapshot: the roles remove entries of inactive profiles anyway.

Two decisions here are deliberate:

• Dropping a profile is not a reset. Removing ospp from the stack without listing it in _reset_hardening_profiles keeps its applied values where no remaining profile manages them. Rolling back is an explicit act.
• baseline cannot be reset. It carries cryptographic policy and the base layer every assessment relies on. Removing it would leave a host in an undefined state.

Several Profiles and Drift

When several profiles are reset at once, they are processed in reverse application order, so the snapshot of the earliest applied profile decides the final value.

This is also where drift becomes visible. Suppose ospp was applied after extra, and both touch the same item. When ospp was activated, its snapshot should have recorded the value that extra had configured. If the snapshot shows something else, someone changed the host between the two deployments. The engine reports that as a conflict:

_profile_reset_conflicts:
  - category: sysctl
    key: net.ipv4.conf.all.rp_filter
    profile: ospp
    snapshot: '0'
    expected: '1'

By default conflicts are reported and the reset continues. With _profile_reset_conflicts_fail: true the run stops, which is the right setting for environments where unexplained changes need an investigation, not a quiet restore.

Safety Nets Around the Reset

Resetting touches the same fragile areas as hardening, so the same safety nets apply:

• /etc/fstab is validated with findmnt --verify before the run ends. A broken fstab is discovered by Ansible, not by the next boot.
• Boot changes are checked before a reboot. Before enabling FIPS mode triggers its one-time reboot, the role checks that the initramfs actually contains the FIPS module, so the host never reboots into a state it cannot boot from.
• The engine is tested without a host. The merge, the reset plan and the conflict detection live in a pure compute step that controller-local tests exercise with a set of profile-stack cases plus dedicated suites for reset plans, boot arguments and fstab handling (see Layered Testing for Ansible Hardening Code).

Conclusion

Reversible hardening is less about rollback code than about three habits: treat profiles as data with explicit merge rules, record every place where one baseline overrides another, and capture the pre-change state exactly once, before the first change.

With those in place, "can we take this profile back off that host?" stops being an investigation and becomes one line in the host’s variables.