Every maintenance window is an attack window
Setup Mode logs but stops blocking; the maintenance kernel unloads Root Lock. How to make changes without leaving a hole.
Overview: Every maintenance window is an attack window. In Setup Mode the kernel logs but stops blocking. On the maintenance kernel, Root Lock by HeartSuite is not loaded at all.
These guides cover how to make changes without leaving a hole an attacker can use. The Dashboard shows the current protection state — including Lockdown status — and the Suggested Next Step throughout maintenance.
Maintenance is a time period during which you temporarily step out of Lockdown to make changes. It is not a separate mode. Root Lock has two modes: Setup Mode and Lockdown.
During maintenance you either switch to Setup Mode (the kernel logs but stops blocking) or boot the maintenance kernel (Root Lock is not loaded). The Dashboard’s Maintenance ([m]) detects whether the immutable seal is active and opens the matching path.
Installing packages, applying patches, and editing configuration happen in Setup Mode once the window is open — that is where blocking is off and logging stays on. After the first Lockdown, opening that window takes a console GRUB pick: Maintenance: unseal and return to Root Lock. The seal lifts automatically and you land back in Setup Mode on the Root Lock kernel. A one-reboot switch with no GRUB is only when the strip already says Lockdown not applied.
- Seal not applied. Type
YES and reboot once. You stay on the Root Lock kernel in Setup Mode. No GRUB pick. - Seal applied (the usual path after the first Lockdown). Physical or serial console is required to unseal. Reboot and select Maintenance: unseal and return to Root Lock. That is two reboots before the window is open. Installs and edits after that are over SSH in Setup Mode.
The Maintenance grid button is shown in Lockdown. Keyboard [m] also works in Setup Mode after you have unsealed.
Which change is this?
Installing packages, replacing program files, and updating Root Lock itself are different jobs, and they take different paths out of Lockdown.
In this section
- Protecting During Maintenance — Console unseal after Lockdown (SSH is not enough for the GRUB pick). Then install or edit over SSH in Setup Mode. Ansible can run after the window is open; it cannot lift the seal.
- File Backup and Versioning — Automatic versioned backups on the Root Lock kernel. Under Lockdown the kernel is intended to keep other programs off those versions. Restore any earlier version from Backup.
- Cache Adjustment — The allowlist cache is an LRU window the Dashboard expands for you. Manual sizing is optional.
- Restricting Kernel Module Loading — Narrow kmod’s file access before Lockdown. Seal prep can auto-narrow directory grants under
/lib/modules. - Updating Root Lock — Unseal if Lockdown is applied, then run the bundle from a terminal in Setup Mode. Type
YES for one stock boot. The default stays Root Lock.
1 - Keep a gate up while you change the system
After Lockdown, Maintenance unseals from the console and returns you to Setup Mode. How to shorten the unprotected period, keep a recovery path, and lock down again.
Overview: Every maintenance window is an attack window — blocking is temporarily suspended, and anything an attacker can reach during that period is unprotected.
Maintenance is the period when you temporarily reduce Root Lock by HeartSuite’s protection to install packages or edit files. The Dashboard’s Maintenance ([m]) guides you from the safety checklist through re-engaging Lockdown. Replacing the Root Lock kernel is a different path: Updating Root Lock.
After Lockdown, unsealing is the console step. You select Maintenance: unseal and return to Root Lock at the boot menu. The seal lifts automatically, and the machine returns to the Root Lock kernel in Setup Mode. You do not stay on the maintenance kernel to remove flags by hand. Package installs, configuration edits, and Ansible stay on SSH once you are back in Setup Mode.
A one-reboot switch that stays on the Root Lock kernel applies only when the strip already says Lockdown not applied — the seal is missing. That is not the usual path after a completed Lockdown.
Starting maintenance
From the Dashboard in Lockdown, select Maintenance ([m]). The Dashboard detects whether the immutable seal is active and presents the correct path.
Safety checklist
Before any mode change, Maintenance presents a safety checklist. The Dashboard auto-detects system state where possible and shows the status of each item:
- Network isolation — disable network interfaces or restrict firewall rules to prevent remote access during maintenance
- Server processes — shut down daemons (e.g., web servers) to close attack vectors
- SSH access — no root login, key-based auth only, source IP restriction
The Dashboard shows green checkmarks for items that pass and amber warnings for items that need attention. Press [c] Confirmed to proceed or [s] Skip to continue without completing the checklist. If you skip, the Dashboard displays a persistent reminder throughout the maintenance period — it does not disappear until you re-engage Lockdown.
The safety checklist matters most when you are about to lift the seal. While you are in Setup Mode with the Root Lock kernel still loaded, logging and backups continue.
After Lockdown: unseal from the console
This is the path when Lockdown is applied. Physical or serial-console access is required to pick the boot-menu entry (keyboard and monitor, a serial port, or your cloud provider’s serial console — AWS EC2 Serial Console, GCP Serial Console, Azure Serial Console, DigitalOcean Console). Confirm that access before you start. You cannot unseal from SSH. After the seal lifts, SSH and Ansible are how you install packages and edit files.
After the safety checklist, Maintenance tells you to reboot from the console. It does not offer [r] Reboot on this path — the boot-menu choice has to happen at the console.
- Open the console and restart the machine there.
- At the boot menu, select Maintenance: unseal and return to Root Lock. Do not select the branded Root Lock kernel.
- The seal lifts automatically (
HS_unlock.sh). The machine restarts on its own and returns to the Root Lock kernel in Setup Mode. - On the serial console, press Enter when you see Press Enter to start. (see Lockdown).
The boot menu appears a second time during that automatic return. Let it be: reboot is already in motion and there is nothing to select.
You are then in Setup Mode on the Root Lock kernel:
- Blocking is off; logging and backups are on.
- New activity appears in the review queues.
- Maintenance (
[m]) is hidden — you can already install software and edit files.
Make your changes — install packages and edit configuration. When finished, lock down again from Lockdown ([l]). Review and approve the new queue items before you type YES. The activation flow is in Lockdown.
To replace Root Lock itself, do not wait on this kernel for the bundle. From a terminal in this Setup Mode window run bash heartsuite-install.sh and type YES. See Updating Root Lock.
If you accidentally select the Root Lock kernel at the first boot menu instead of the Maintenance entry, the Dashboard detects that and sends you back to reboot and select the correct entry.
Warning
Between selecting the Maintenance entry and the automatic return, Root Lock is not loaded. The safety checklist is the gate for that interval.
When the seal is not applied
If the strip says Lockdown not applied, Maintenance offers a switch to Setup Mode that stays on the Root Lock kernel. Type YES (case-sensitive). The Dashboard then offers [r] Reboot.
After that reboot:
- Root Lock switches from blocking to logging only
- The Root Lock kernel remains active
- Backups continue running
- The existing allowlist is preserved
- New activity is logged, not blocked — it will appear in the review queues when you lock down again
This path is for an unfinished or drifted seal, not for a host that already shows Lockdown applied.
After the window is open
Once you are in Setup Mode, SSH and Ansible can install packages and edit files on that host. The console trip is only to lift the seal.
- One host, many services. One unseal covers every program on that machine.
- Many hosts already in Lockdown. Ansible cannot lift the seal. The official
heartsecurity.root_lock role leaves mode unchanged when hs_state is unset or setup. In-place patches still need the console path on each sealed host. For a fleet, reprovision from an updated image instead — see Enterprise Adoption Guide and Central Policy. - Detaching the disk. Stopping a cloud VM and attaching its volume to another instance is hypervisor access, not a supported patch procedure. Treat it as the same class as serial-console access: break-glass, and restrict it in cloud IAM. See Circumvention and recovery.
What to install
The maintenance window is how you add new software after Lockdown. Install the packages you will keep. Compilers and package-install helpers that executed only for this window do not belong on the allowlist unless they must stay on the host — do not approve them. See Allowlisting Basics.
When the change is done, review the new queue items, then lock down again from Lockdown ([l]). Do not leave the host in Setup Mode to install tools you will not keep.
Manual recovery outside Maintenance
When Lockdown makes files immutable using chattr +i, those flags are stored at the filesystem level and persist across reboots — including a reboot that reaches the maintenance kernel.
If you attempt to modify a file that was made immutable during a previous Lockdown session, you will encounter an error such as “could not open file; errno:1.”
The console Maintenance entry runs HS_unlock.sh for you. For recovery outside the Dashboard, run HS_unlock.sh.
2 - Backups that ransomware cannot delete
Every write in a protected directory is versioned before it lands. Under Lockdown, other programs are not intended to reach those versions.
Overview: Allowlisting controls what programs can execute, but an approved program that malware takes over can still write files. Ransomware running inside an approved process can encrypt whatever that process can reach.
Modern ransomware targets backup systems first — shadow copies and backup agents are typically deleted before files are encrypted.
Root Lock by HeartSuite creates a versioned backup every time a file in a protected directory is written. Versioning runs on the Root Lock kernel in both Setup Mode and Lockdown. Backup is a recovery store, not a prevention control: it does not stop the first write.
Delayed activation does not change that rule. A malicious sleeper is an approved program that behaves as expected for a long time, then encrypts or overwrites files it is already allowed to write. Root Lock still versions every write in a directory you selected on Backup ([b]). The default is /home.
HeartSuite Joint File System (HJFS) versions every program data file with no directory list. See The malicious sleeper attack.
Under Lockdown, the kernel keeps other programs — including malware running as root — off those backup versions. That gate is absent on the maintenance kernel. Changing which directories are protected requires a maintenance window.
Automatic versioning
Root Lock monitors the protected directories you select. When any file in those directories (including subdirectories) is written, Root Lock creates a new versioned backup before the write completes. Versioning begins from first boot, before you have reviewed a single item.
Enterprise backup tools back up on a schedule — hourly, nightly, weekly. An attack that completes between backup windows has nothing to recover from.
Root Lock backs up on every write. There is no window.
Other security tools that offer rollback on Linux — including endpoint tools with a rollback feature — rely on volume shadow copies or scheduled snapshots. The same gap exists: an attack that completes between snapshot intervals has nothing to recover from.
CVE-2024-40711 — Veeam Backup & Replication, unauthenticated RCE — shows the sharper problem: the backup tool itself is the target. An attacker who reaches a Veeam host can execute code without authentication, destroy backups, then encrypt production files. Root Lock’s backups have no running agent to exploit.
By default, /home is configured for backup. You can add or remove directories from the Dashboard’s Backup.
Configuring protected directories
From the Dashboard, select Backup ([b]). Backup has two tabs: Configure and Restore. Configure shows which directories are protected and when they were last versioned.
From Configure you can:
- Add a directory (
[a]) — protect additional directories (for example /var/www, /etc, /usr/lib) - Remove from backup (
[r]) — stop backing up a directory. Existing versions are retained until the retention window expires.
[n] on Backup is Cancel, not add.
Recommended directories include those containing user documents, executable files, configuration, and shared libraries. Avoid high-churn directories like log directories — backup creates a new version on every write.
Backup is optional. You can remove all directories, disabling backup entirely. Lockdown does not require backup to be configured.
Restoring file versions
If a file is compromised — for example, encrypted by ransomware — Backup lets you browse version history and restore any previous version of any file in a protected directory. On the Restore tab:
- File-first (
[f]) — navigate by directory and file, then view versions of the selected file - Timeline (
[t]) — navigate by date, showing all files modified on a given day
To restore a single file, select it and choose the version to restore. Each version shows its timestamp and file size.
For ransomware recovery where many files were modified on the same date, use Timeline ([t]), press [d] to filter by date, review the affected files, and press [b] to batch restore all of them in one operation.
Lockdown and backup
When Lockdown is active, the backup configuration file is sealed — no user or program, including root, can add or remove directories. This prevents an attacker who compromises a running process from silently disabling backup. To change the backup configuration, enter a maintenance period first (see Protecting During Maintenance).
Backup encryption
Root Lock backup files are versioned filesystem copies. They are not encrypted by Root Lock. If your environment requires data-at-rest encryption — for example, to meet GDPR, HIPAA, or PCI DSS requirements — configure full-disk encryption (dm-crypt/LUKS) at the OS level. LUKS encryption covers the backup files automatically, since they reside on the same filesystem as the rest of the host.
CLI access for scripting and automation
For scripting and automation that runs without the Dashboard:
# hs-backup-config-manager list
# hs-backup-config-manager add -d /var/www
# hs-backup-config-manager del -d /home
# hs-version-manager list /home/user/document.txt
# hs-version-manager replace /home/user/document.txt <token>
The Dashboard is the supported path for normal use.
3 - When 255 allowlist slots is not enough
The Dashboard expands the kernel allowlist cache up to 255. Larger allowlists stay valid; the cache keeps the most recently used entries.
Overview: Root Lock by HeartSuite caches allowlist entries in kernel memory for lookup speed. The cache is an LRU window, not a limit on how many programs you may approve. The Dashboard expands that window toward your allowlist size, up to 255 entries. Allowlists larger than 255 stay valid; the kernel evicts the least recently used cache slots.
Manual sizing is optional. You do not have to prune the allowlist when it grows past 255.
Automatic cache expansion
On startup and every state refresh, the Dashboard compares the size of your allowlist against the current kernel cache size. If the allowlist is larger, the Dashboard silently expands the cache — up to 255 entries. The minimum cache size is 10.
This runs in the background on the Dashboard’s normal 60-second refresh cycle. You do not need to invoke a CLI tool or change a setting.
When the allowlist is larger than 255
Auto-expansion stops at 255. The extra allowlist entries remain in force; they are not refused. The kernel keeps the most recently used 255 in the cache.
Pruning unused programs in Allowed ([a]) is hygiene, not a hard stop. After you remove entries, the next Dashboard refresh can shrink the working set the cache has to hold.
There is no Dashboard warning of the form “Allowlist has 312 entries but kernel cache max is 255.”
CLI access for scripting and automation
For scripting and automation that runs without the Dashboard, set the cache to a size between 10 and 255 with the on-disk tool:
# /.hs/sys/hs-APO-cache-size 128
Docs and older notes may say hs-cache-size. That is the glossary name; the binary on disk is hs-APO-cache-size.
The Dashboard is the supported path for normal use.
4 - Stop kmod from loading just any module
If kmod can execute, limit which module files it may read before Lockdown. Directory grants under /lib/modules are the real risk.
Overview: Root Lock by HeartSuite does not load kernel modules through a separate init_module gate. What kmod may load is what it may read. If kmod, modprobe, or insmod can execute and can read a .ko file, that module can be loaded.
If your hardware requires kmod at startup to load device drivers or filesystem modules, kmod must have an allowlist entry. Restrict that entry’s file access to only the specific modules it needs before engaging Lockdown. An allowlisted kmod with unrestricted file access can load any module on the machine.
If kmod, modprobe, and insmod have no allowlist entries, Lockdown refuses to execute them. You can skip the rest of this page.
When kmod is allowlisted
Some hardware configurations require kmod at startup to dynamically load drivers or filesystem modules the system needs to boot. Once kmod has an allowlist entry, it can execute — and without further restriction, kmod’s file access permissions determine which modules it can load.
The hardening step is to narrow those file access permissions to the specific module paths kmod legitimately needs. If kmod tries to load a module outside its permitted paths, Root Lock denies the file access in Lockdown before the module can be read.
An allowlisted kmod with directory-level read under /lib/modules can open module files that were never observed during Setup Mode.
Narrow file access before Lockdown
Do this before you type YES on Lockdown. Once Lockdown is active, allowlist entries are sealed. Changing them takes a maintenance window.
When kmod’s startup activity appears in the File Access queue ([f]) during Setup Mode, approve individual .ko paths rather than directory-level access. Approving a directory grants read access to everything under it — including modules not present during observation.
If directory grants under /lib/modules are still present when you open Lockdown ([l]), seal prep auto-narrows them. That panel is an advisory, not a YES gate. [m] on that panel undoes the narrowing — it is not Maintenance. Under Lockdown the same inventory is read-only.
Leftover grants after auto-narrow belong in Allowed ([a]) or File Access ([f]), not a CLI as the normal path.
After narrowing, reboot and confirm the machine starts with no unexpected kmod denials in the review queues. Then activate Lockdown ([l]).
What stays sealed after Lockdown
After Lockdown engages:
- Allowlist entries are sealed — kmod’s entry cannot be modified while Lockdown is active.
- Startup scripts are sealed — system-wide shell configuration, systemd unit directories, and cron. Attackers cannot insert scripts that would run before Lockdown re-engages on the next boot and expand kmod’s permissions.
Lockdown does not independently refuse init_module. The restriction is the program allowlist plus file access on module paths.
Per-user shell profile coverage
Lockdown seals system-wide shell configuration — /etc/profile, environment defaults, and cron — preventing an attacker from planting scripts that run at the next boot and expand kmod’s permissions before Lockdown re-engages. Per-user profile files (~/.bash_profile, ~/.bash_login, ~/.profile, ~/.bashrc, ~/.inputrc) are not covered automatically because the correct set depends on your user configuration.
If specific user accounts need that coverage, enable the commented-out entries for those users in HS_lockdown.sh before engaging Lockdown.
5 - Apply an update without dropping Lockdown blindly
Unseal if Lockdown is applied, then run the bundle from a terminal in Setup Mode. Type YES to take one stock boot. The default stays Root Lock. You land in Setup Mode on the new kernel.
Overview: A Root Lock by HeartSuite update is delivered as a single self-extracting bundle (heartsuite-install.sh) that replaces the kernel and its userspace tools in one operation.
What an update changes
- The Root Lock kernel
- HeartSuite userspace tools (
activate_HS, lockdown_HS, Secure Script Launchers, setup scripts) - The Dashboard files under
/opt/heartsuite/ - GRUB configuration, so the new kernel becomes the default boot target
It does not wipe user data, existing allowlist entries, or backup files. A complete overwrite still re-runs first-run setup (extra reboots) before the Dashboard returns. After the new kernel is up, new programs appear in the review queues for you to approve.
Why the update needs a stock boot
The installer will not overwrite Root Lock while that kernel is booted. It checks uname -r. Setup Mode is still the Root Lock kernel. You can copy the bundle there, and you can install packages and edit configuration there.
You can also start bash heartsuite-install.sh there from a terminal. The installer asks you to type YES (case-sensitive). That sets the next boot only to the original distro kernel (stock) or Maintenance, keeps Root Lock as the default, and continues the update after that boot. You do not pick a menu entry, and you do not run the installer a second time.
If the session is not a terminal — for example curl … | sudo bash — the installer prints a one-shot command such as grub-reboot '…' && reboot for this machine instead of asking YES. Copy that command; do not invent a number. After that boot, run bash heartsuite-install.sh again. If the installer cannot name an entry, it tells you to open the console and choose stock or Maintenance at the boot menu.
The first-install command curl -fsSL https://get.heartsecsuite.com/get-heartsuite.sh | sudo bash is for a host that does not yet have Root Lock. On a host already running the Root Lock kernel it downloads the bundle and then follows the non-terminal path above.
If Lockdown is applied (the strip says Lockdown applied), unseal first. The installer cannot set the next boot while /boot is sealed.
- Already in Setup Mode on the Root Lock kernel. Skip Unseal. Place the bundle, verify the checksum, and run
bash heartsuite-install.sh from a terminal. Type YES. - Already on the original distro kernel or the maintenance kernel, and the strip does not say Lockdown applied. Skip Unseal and skip
YES. Run the installer. - On the maintenance kernel but the strip still says Lockdown applied. The immutable flags are still on the files. Unseal first.
On the stock or maintenance boot, Root Lock is not loaded: it does not block programs, does not log, and does not take backups. Choose a time when that gap is acceptable.
Before you begin
- Unseal first if Lockdown is applied. From the Dashboard, open Maintenance (
[m]) and follow the sealed path in Protecting During Maintenance. You must land in Setup Mode before the installer can set the next boot. - Verify the bundle. Compare the SHA-256 of
heartsuite-install.sh against the published checksum before running it. - Physical or serial-console access if the installer cannot name a boot entry, and for recovery if the new kernel does not boot. You do not need the console only because you are in Setup Mode.
Update procedure
Place heartsuite-install.sh and heartsuite-install.sh.sha256 on the system, typically by scp into /root/. In Setup Mode that copy is allowed. Under Lockdown it may be denied — unseal first.
If Lockdown is applied, open Maintenance ([m]) and complete the unseal path. You should be in Setup Mode on the Root Lock kernel before the next step.
Verify integrity:
sha256sum -c heartsuite-install.sh.sha256
Expected output: heartsuite-install.sh: OK
Log in as root over SSH or the serial console (AWS EC2 Serial Console, Linode LISH, Hetzner, and others). On failure, inspect /var/log/heartsuite/install.log on the serial console.
Run the installer from that terminal:
bash heartsuite-install.sh
If uname -r still names a Root Lock release, type YES when asked. The machine takes one stock or Maintenance boot and the installer continues from disk. The default stays Root Lock. If you are already off the Root Lock kernel, there is no YES step.
The installer applies the update and, by default, reboots into the new Root Lock kernel. You land in Setup Mode.
If new programs appear, they show in the review queues. Approve them through the Dashboard, then re-engage Lockdown ([l]) if it was active before the update.
Many hosts
The in-place bundle is per host. After Lockdown, each sealed host still unseals from the console before the installer can set the next boot — /boot is sealed.
For many locked hosts, reprovision from an updated pre-configured image instead of running the bundle on each live machine. That path is equivalent for support when the image contains a published bundle. See Kernel Support Policy and Enterprise Adoption Guide.
If the update fails
If the new Root Lock kernel does not boot, select the previous kernel from the GRUB menu. Physical or serial-console access is required for this step.
Both the previous Root Lock kernel and the maintenance kernel remain available as recovery entries. Contact HeartSuite support at support@heartsecsuite.com and include the contents of /var/log/heartsuite/install.log in your message — we’re happy to help you recover.