25 years of OpenBSD Security Tools: syslock and sysunlock

If you missed the overview post, you can see it here. This one is about managing immutable and append-only files on *BSD, Linux, and macOS.

Immutable and Append-Only Files

BSD-derived operating systems (including macOS) and Linux both support the concept of files being made immutable, so that neither their contents nor attributes can be changed. They also both support files being made append-only, so that the existing contents cannot be changed except by adding more data to the end. They do it in slightly different ways.

BSD Implementation

On BSD-derived systems, these features are controlled using file system flags which have "system" and "user" variants; the former can only be set and unset by the root user, while the latter can be set and unset by any user on files they own. The flags are changed using the chflags command, and their names are schg (system immutable), uchg (user immutable), sappnd (system append-only), and uappnd (user append-only). BSD-derived systems also take the further step for the system-level flags that they can be set but not unset after the system has booted from single-user mode to multi-user mode. This is controlled by the system security level (kern.securelevel) which is raised automatically by the init process from 0 ("Insecure Mode," also known as single-user mode) at initial boot to 1 ("Secure Mode," the default multi-user mode).  The system security level may also be raised with the sysctl command but cannot be lowered except by shutting the system down to return to single-user mode (securelevel=0), which shuts down system daemons and drops network connections, leaving the system accessible only by the system console until it reboots. The effect of this is that even the root user cannot modify the contents or attributes of files with schg (and can only append to files with sappnd) without shutting down the system.

Linux Implementation

On Linux, by contrast, only the root user can set or unset immutable or append-only attributes, using the chattr command with +i/-i or +a/-a to set or unset immutable or append-only attributes, respectively. But there is no system security level that prevents unsetting these attributes at any time after they've been set. The Linux attributes are equivalent to the BSD user flags, but limited to the root user. I have used syslock less on Linux than on OpenBSD, but have used it fairly broadly on Proxmox and Kali Linux.

macOS Implementation

On macOS, the BSD flags are present but by default the system is always in "Insecure Mode" (securelevel=0), and Apple has added additional file flags, notably restricted, which it uses on operating system commands and libraries as part of its "System Integrity Protection" (SIP) feature added in OS X El Capitan (10.11) in 2015. The restricted flag cannot be unset even when the system securelevel=0, but only when the system is booted into Recovery mode. While the restricted flag is, like the immutable and append-only flags, managed by the chflags command, it is not supported by my tools and it's just mentioned here to note that macOS has implemented a very similar capability in a different way. I have the least experience in using syslock on macOS, and because of that and the fact that system binaries are protected by this alternative mechanism, I use it less broadly there.

BSD Security Levels

When I first learned about file system immutable flags, the default system startup file on OpenBSD named /etc/rc.securelevel contained lines to set the system securelevel to 1, but that now happens automatically in the boot sequence by the init process, and the default rc.securelevel example file in /etc/examples consists only of comments and is no longer installed by default. All of my systems have that config file installed to set the securelevel, for reasons which will be explained shortly.

OpenBSD supports two other settings for kern.securelevel besides 0 and 1, which are -1 ("Permanently Insecure Mode") and 2 ("Highly Secure Mode"). Permanently Insecure Mode prevents the securelevel from being raised to 1 automatically; it instead goes to 0 and remains there.  Highly Secure Mode features all of the restrictions of Secure Mode, plus also restricts changes to host firewalls with OpenBSD's pf packet filter, allowing only changes to what IP addresses are in tables but no changes to filtering or NAT rules. The details of each level are documented in the OpenBSD securelevel man page. These restrictions are intended to reduce the impact and blast radius of both system compromises where an attacker gains root and to reduce the impact of administration errors.

Security Control, Administrative Safeguard, or Security Theater

The latter goal -- reducing the impact of administration errors -- is one that the BSD, Linux, and macOS default settings all support, but the former goal -- reducing the impact and blast radius of system compromises where an attacker gains root privileges -- is only evident for the BSD system immutable and append-only flags, where even the root user cannot unset them so long as the system is in Secure or Highly Secure mode. Some have argued that these flags are also mere speed bumps or error prevention (or "security theater"), on the grounds that they are easily bypassed, which can be done by returning the system to Insecure Mode. There are two main ways for a user with root access to do that, which are (1) using console access to access a root shell in single-user mode after shutdown, which may not be particularly easy as a remotely connected attacker likely does not have console access, or (2) modifying configuration files that allow command execution before the system raises the system securelevel, and rebooting the system. There are many configuration files and commands executed on the system during the boot sequence while the system is in Insecure Mode, and if any of those can be modified to either execute commands or prevent the system from going into Secure Mode, there is a path to unsetting the system immutable and append-only flags for the attacker (at the cost of a potentially noisy reboot). (I have made use of this path myself in the past while testing immutable flags and getting myself stuck.) It's because many files would have to be set immutable to close off that second reboot path that some have called even the system flags "security theater," but I think it is both feasible and it can be a genuine security control. The key is making it practical to lock enough of the right files, which is the problem I've tried to solve with these tools.

Origins of syslock/sysunlock

My syslock/sysunlock tool (two opposite functions in a single perl script, from here on I'll generally just refer to syslock except when the distinction matters) is designed to make the management of all of these implementations of immutable and append-only file system controls feasible, straightforward, and usable, at least as an error prevention method and at best as a security control. I originally wrote it after coming across a simple shell script of the same name by George Shaffer, but it now looks quite different, supporting system and user immutable and append-only flags on OpenBSD and macOS, and Linux's near equivalent (+i/-i and +a/-a). My recommendation for anyone starting out on a BSD system is to begin with the user flags, which are trivial to unlock and cause no permanent damage if you need to make changes quickly. Once comfortable with the group structure, you can consider adding system flags for files that rarely need to change. I'll describe that group structure next.

syslock Groups

The main feature of syslock that makes it usable is that it is configured to place lists of files and directories into groups, and those groups can be locked or unlocked with a single command (syslock -g <groupname>). The groups are defined like tags associated with a list of files and directories, so that a given file or directory can be in multiple groups. The default configs supplied with the tool include group names such as etc (files in /etc), etcrare (files in /etc that are rarely modified), and fstab (/etc/fstab gets its own group as it's a painful file to clobber by accident). There are relatively self-explanatory group names like binaries, libraries, and system, and then there's presecure, which covers the files and directories that are potential targets for someone trying to find a way to bypass system immutable flags (e.g., startup scripts like /etc/rc, /etc/rc.local, /etc/rc.d, and others that may be less obvious like /etc/sysctl.conf).  (There is no presecure group defined in the Linux or macOS example configs.) For BSD and macOS, you can choose to make user immutable or system immutable your default, but you can also use groups to explicitly call out a set of files as the opposite of your default, with the group names schg and uchg (and similarly with sappnd and uappnd groups). These groups are specially handled so that they can be specified in combination with another group name (e.g., etc:uchg or acct-logs:uappnd) to identify the files and directories that are in both.

Since it is possible to enable the schg flag in Secure Mode but not to disable it, by default neither syslock nor sysunlock will touch those files while in Secure Mode (sysunlock can't unlock them), but the -f (force) option to syslock will lock them. A group can have both a system and a user subset of files regardless of what default is configured. For example, if your default is schg but you want some commonly modified files in /etc to be uchg, you could put them in that group along with the etc  group. Then, if you unlocked files in Insecure Mode with -g etc, both the schg and uchg files in the etc group would be unlocked, while if you used sysunlock -g etc in Secure Mode, only the uchg files would be unlocked. If you were in Insecure Mode but only wanted to lock the uchg files in the etc group, you can use -g etc:uchg.

The intention of groups is to provide a mechanism for unlocking specifically what is needed to perform a specific task. The default and sample config files supplied with the tool include groups for other tools that will be covered in this series of blog posts, including reportnew (blog post June 16), rsync (for rsync-tools, blog post June 9), and sigtree (blog post June 19), as well as for process accounting log files (to make them append-only), mail servers, web servers, and DNS servers (the latter two in the BSD default config only). Log files and DNS servers provide two examples for where you'd want to use different flag-specific subgroups. For log files, you want the active log file to be append-only, but you want the rotated and archived log files to be immutable; you also want them to use user rather than system flags if the logs are subject to rotation via newsyslog or other log rotation mechanism. Thus the sample configs put the live process accounting log in the uappnd group and the rotated process accounting logs in the uchg group. The rotation process needs to unlock both of those subgroups before rotation, and lock them again after rotation. For DNS configuration files, you might want to lock your zone files with schg if they don't change frequently and aren't changed by any automation, but if you use automated DNSSEC signing, you'd lock the signed zone files with uchg so that they can be unlocked before signing and relocked after signing. It's worth noting that logging to a separate machine is a much better security control than using uappnd (or Linux +a) flags on log files--but I do both.

Overlapping Groups

Note that it is possible to define groups that overlap with each other in various ways, and while this is generally acceptable and unlikely to cause any issues on Linux or if only a single type of flag is used on BSD or macOS, it can create issues if overlapping groups have a mix of flag types such that multiple flags get set on any files or directories. The default and sample configs do not contain any group definitions that create this problem, and syslock will generate warnings if it detects cases of conflicting flag types in the configuration, but will not detect all possible cases (e.g., where symlinks are involved).

OpenBSD-specific: KARL Features

There is also an implicit group of files identified in the config file by a leading "!" prefix character, which designates system files that are part of OpenBSD's Kernel Address Randomized Link (KARL) feature, where the kernel and key binaries (currently libc, libcrypto, ld.so, sshd, sshd-session, sshd-auth, and ssh-agent -- the list may increase between releases) are relinked in random order at reboot.  These files are locked or unlocked by using a -s (for system) option. (This was perhaps a poor choice of option letter and name, as it is distinct from the "system" group which is intended to capture key operating system files and what needs to be unlocked for upgrading and patching, though I typically unlock everything before upgrading or patching. This is just to ensure that nothing the system needs to install is blocked, which can leave a system in an inconsistent and not fully operable state, but the "system" group should actually work for this purpose.) Since I use system immutable flags as my default on most systems, and on the presecure group on all systems, my practice for patches and upgrades is to shut the system down (enter Insecure Mode), unlock everything (sysunlock with no arguments), perform the upgrade (with an extra step) or patch, then lock everything (syslock with no arguments), then unlock what's needed for Karl (sysunlock -s), and then exit, which starts up system daemons and networking, and returns to Secure Mode, without rebooting the kernel or resetting the system uptime counter. In my rc.securelevel file I have the following line:

echo -n ' running syslock'; (/bin/sleep 10; /usr/local/bin/syslock -swf) &

The -s and -f options have already been explained, but -w means to wait for KARL relinking to complete before locking, so as not to prevent that process from occurring. Typically, I do not unlock with -s for most instances when I shut down to make changes, and so I do prevent KARL relinking in those cases, and it doesn't lead to any system inconsistencies--it just preserves the previously established link ordering.

The extra step I mentioned above for an upgrade is that I comment out the line in rc.securelevel that re-locks the system because at the completion of a sysupgrade process there will be a reboot after which I will want to make many additional file changes using sysmerge to update system configuration files, update packages I have installed, remove unused binaries and old libraries, and so forth.  I then remove the comment and reboot again. (Also, if a patch with syspatch rebuilds the kernel, I'll reboot with the new kernel after the return to Secure Mode and the KARL process completes.)

One side note is a little trick I use on my OpenBSD firewall to avoid unnecessary disruption to traffic when doing patching and software updates that require unlocking system immutable flags. The trick is that shutting down to Insecure Mode doesn't impact traffic routing or packet filtering, but running netstart when you exit back to Secure Mode does. (Disruption is unavoidable if it's a kernel patch that requires a full reboot.) The trick is that I've modified my /etc/rc (something I generally avoid doing for ease of upgrades, so I keep the original) by adding one line before and two lines after the call to netstart:

if [ ! -f /var/run/no_netstart_resume ]; then

    sh /etc/netstart

    echo "bypassing netstart"

    /bin/rm -f /var/run/no_netstart_resume

With this in place, I touch /var/run/no_netstart_resume before I exit, and the system restarts with no network disruption.

Audit Feature

Both syslock and sysunlock have an audit (-a) option, which will report which files are currently not in the expected state. That is, syslock -a will tell you all the files which should be locked per the config but are unlocked, and sysunlock -a will tell you all the files which should be unlocked per the config but are locked. This can also be applied to any specific group with -g <groupname>. If used on a BSD system in Secure Mode, the audit can be restricted to what would actually be changed at the current security level using the -o (operational restrictions) option. A -q (quiet) option will suppress all output and just return 0 for success or 1 (error) for failure; this was created to allow a check to see if all files in a particular group are unlocked for an installation to occur for my install.pl tool (to be covered in a June 12 blog post). With -q, the audit will finish and return at the first discrepancy found.

Path Prefixes

The configuration file syntax allows listed paths to use three other prefixes in addition to the OpenBSD-specific "!" referenced above. These prefixes, which only have effect on directory paths (and generate a warning but are otherwise ignored on other file types) are:

+  Do not recurse through subdirectories.  (No prefix, the default, means lock the directory and every thing in it, recursively.)

-  Do not lock the directory itself, just its contents.

=  Lock the directory and its file contents, but not subdirectories.

These cannot be used in combination (though some combinations, like =-, might conceivably be useful), and while I initially made use of the first two path prefixes in my configs, they ended up not being particularly useful with the patterns of groups I developed. I recently added the = path prefix to address a Linux-specific case.

Linux-specific: Bootloader Considerations

On many Linux systems (notably on Proxmox which uses Debian), the grub bootloader rewrites files in /boot/grub, (specifically grub.cfg and grubenv) on reboots. If those files are immutable, the reboot will fail to update those files. The default Linux config creates a grub group for /boot/grub and its contents, and uses the = path prefix on a separate group for /boot and its other subdirectories. The grub group can then be unlocked before a reboot, while the broader group is unlocked for kernel updates, and both locked again afterward. The grub group unlocks slightly more than strictly necessary (everything in /boot/grub rather than the directory and the two specific files), but handles the common case cleanly.

Use of Large Language Models (LLMs)

I've made use of LLMs, initially for performing security assessments with suggested improvements that I'd implement selectively and by hand, but subsequently for working out the details of design for prospective changes, writing code, and identifying the causes of bugs. Each of these capabilities has significantly improved since the initial use for code assessments in the summer of 2025. Specific capabilities implemented by Claude include adding append-only flag support, improving error messages and warnings, code refactoring, enhancing the audit features and adding the -o option, and drafting the Github README. Most recently, while I was in the process of writing this blog post, I revisited an issue I had run into involving overlapping groups with differing flags, and thought that a new path prefix that ignored directories might be a good way to handle it. In addition to that specific use case, for which Claude suggested the use of = (to mean "lock the directory and files at the same level"), I asked whether any combinations of existing path prefixes might be useful or if any other new prefixes might make sense. Claude suggested that -= or =- might be meaningful and useful (don't lock the directory itself, lock the files in it but not the subdirectories), but I chose not to add the additional complexity without a specific use case. In the process I identified a bug in how path prefixes were being handled in previous Claude-generated code related to perl taint handling, and Claude fixed it along with the implementation of =.

I've used ChatGPT and Gemini in addition to Claude for security assessments on the code, and they've each identified different issues. While each has found real bugs, my impression is that I've seen more false positives from ChatGPT and Gemini than from Claude, though they've also found real bugs in code written by Claude.

I've found LLMs quite useful for security auditing, refactoring, and error message improvements, as well as writing new capabilities that are relatively straightforward. In the case of implementing append-only flags, it took multiple design, build, and test cycles to get the code to production quality. Overall, forcing myself to explain design choices clearly (and sometimes reconsider them) in the process of using LLMs has been extremely valuable, and has also been a benefit of writing this blog post. I used Claude to identify key topics that should be mentioned in this series, but did not rely on Claude to outline or write the post. Claude and a human editor reviewed this post (and the opening overview post) identify typos and suggest edits.

I'll include a section like this in each post in the series.

Getting Started and Further Reading

The sample config files supplied with syslock cover OpenBSD, Linux (with some Proxmox specifics in comments), and macOS, and provide a reasonable starting point. As noted above, I recommend starting with uchg groups on BSD and macOS, which gives you immediate error prevention value and a chance to discover what needs to be unlocked for your specific regular workflows before committing to more restrictive schg flags. For Linux, there's only one kind of immutable attribute and it's changeable without a shutdown, so you can start immediately with the sample config.

As noted above, the syslock audit feature is integrated with my install.pl tool and syslock groups more generally are a key feature of both that tool and its distribute.pl counterpart, both of which will be covered in a blog post on June 12. The wrap-up post on June 23 will show how all of these tools fit together into a coherent security architecture.

syslock is available on my website and on Github, along with sample configs for OpenBSD, Linux, and macOS.