25 years of OpenBSD Security Tools: syslock and sysunlock

If you missed the overview post, you can see it here. This one is about managing immutable and append-only files on *BSD, Linux, and macOS.

Immutable and Append-Only Files

BSD-derived operating systems (including macOS) and Linux both support the concept of files being made immutable, so that neither their contents nor attributes can be changed. They also both support files being made append-only, so that the existing contents cannot be changed except by adding more data to the end. They do it in slightly different ways.

BSD Implementation

On BSD-derived systems, these features are controlled using file system flags which have "system" and "user" variants; the former can only be set and unset by the root user, while the latter can be set and unset by any user on files they own. The flags are changed using the chflags command, and their names are schg (system immutable), uchg (user immutable), sappnd (system append-only), and uappnd (user append-only). BSD-derived systems also take the further step for the system-level flags that they can be set but not unset after the system has booted from single-user mode to multi-user mode. This is controlled by the system security level (kern.securelevel) which is raised automatically by the init process from 0 ("Insecure Mode," also known as single-user mode) at initial boot to 1 ("Secure Mode," the default multi-user mode).  The system security level may also be raised with the sysctl command but cannot be lowered except by shutting the system down to return to single-user mode (securelevel=0), which shuts down system daemons and drops network connections, leaving the system accessible only by the system console until it reboots. The effect of this is that even the root user cannot modify the contents or attributes of files with schg (and can only append to files with sappnd) without shutting down the system.

Linux Implementation

On Linux, by contrast, only the root user can set or unset immutable or append-only attributes, using the chattr command with +i/-i or +a/-a to set or unset immutable or append-only attributes, respectively. But there is no system security level that prevents unsetting these attributes at any time after they've been set. The Linux attributes are equivalent to the BSD user flags, but limited to the root user. I have used syslock less on Linux than on OpenBSD, but have used it fairly broadly on Proxmox and Kali Linux.

macOS Implementation

On macOS, the BSD flags are present but by default the system is always in "Insecure Mode" (securelevel=0), and Apple has added additional file flags, notably restricted, which it uses on operating system commands and libraries as part of its "System Integrity Protection" (SIP) feature added in OS X El Capitan (10.11) in 2015. The restricted flag cannot be unset even when the system securelevel=0, but only when the system is booted into Recovery mode. While the restricted flag is, like the immutable and append-only flags, managed by the chflags command, it is not supported by my tools and it's just mentioned here to note that macOS has implemented a very similar capability in a different way. I have the least experience in using syslock on macOS, and because of that and the fact that system binaries are protected by this alternative mechanism, I use it less broadly there.

BSD Security Levels

When I first learned about file system immutable flags, the default system startup file on OpenBSD named /etc/rc.securelevel contained lines to set the system securelevel to 1, but that now happens automatically in the boot sequence by the init process, and the default rc.securelevel example file in /etc/examples consists only of comments and is no longer installed by default. All of my systems have that config file installed to set the securelevel, for reasons which will be explained shortly.

OpenBSD supports two other settings for kern.securelevel besides 0 and 1, which are -1 ("Permanently Insecure Mode") and 2 ("Highly Secure Mode"). Permanently Insecure Mode prevents the securelevel from being raised to 1 automatically; it instead goes to 0 and remains there.  Highly Secure Mode features all of the restrictions of Secure Mode, plus also restricts changes to host firewalls with OpenBSD's pf packet filter, allowing only changes to what IP addresses are in tables but no changes to filtering or NAT rules. The details of each level are documented in the OpenBSD securelevel man page. These restrictions are intended to reduce the impact and blast radius of both system compromises where an attacker gains root and to reduce the impact of administration errors.

Security Control, Administrative Safeguard, or Security Theater

The latter goal -- reducing the impact of administration errors -- is one that the BSD, Linux, and macOS default settings all support, but the former goal -- reducing the impact and blast radius of system compromises where an attacker gains root privileges -- is only evident for the BSD system immutable and append-only flags, where even the root user cannot unset them so long as the system is in Secure or Highly Secure mode. Some have argued that these flags are also mere speed bumps or error prevention (or "security theater"), on the grounds that they are easily bypassed, which can be done by returning the system to Insecure Mode. There are two main ways for a user with root access to do that, which are (1) using console access to access a root shell in single-user mode after shutdown, which may not be particularly easy as a remotely connected attacker likely does not have console access, or (2) modifying configuration files that allow command execution before the system raises the system securelevel, and rebooting the system. There are many configuration files and commands executed on the system during the boot sequence while the system is in Insecure Mode, and if any of those can be modified to either execute commands or prevent the system from going into Secure Mode, there is a path to unsetting the system immutable and append-only flags for the attacker (at the cost of a potentially noisy reboot). (I have made use of this path myself in the past while testing immutable flags and getting myself stuck.) It's because many files would have to be set immutable to close off that second reboot path that some have called even the system flags "security theater," but I think it is both feasible and it can be a genuine security control. The key is making it practical to lock enough of the right files, which is the problem I've tried to solve with these tools.

Origins of syslock/sysunlock

My syslock/sysunlock tool (two opposite functions in a single perl script, from here on I'll generally just refer to syslock except when the distinction matters) is designed to make the management of all of these implementations of immutable and append-only file system controls feasible, straightforward, and usable, at least as an error prevention method and at best as a security control. I originally wrote it after coming across a simple shell script of the same name by George Shaffer, but it now looks quite different, supporting system and user immutable and append-only flags on OpenBSD and macOS, and Linux's near equivalent (+i/-i and +a/-a). My recommendation for anyone starting out on a BSD system is to begin with the user flags, which are trivial to unlock and cause no permanent damage if you need to make changes quickly. Once comfortable with the group structure, you can consider adding system flags for files that rarely need to change. I'll describe that group structure next.

syslock Groups

The main feature of syslock that makes it usable is that it is configured to place lists of files and directories into groups, and those groups can be locked or unlocked with a single command (syslock -g <groupname>). The groups are defined like tags associated with a list of files and directories, so that a given file or directory can be in multiple groups. The default configs supplied with the tool include group names such as etc (files in /etc), etcrare (files in /etc that are rarely modified), and fstab (/etc/fstab gets its own group as it's a painful file to clobber by accident). There are relatively self-explanatory group names like binaries, libraries, and system, and then there's presecure, which covers the files and directories that are potential targets for someone trying to find a way to bypass system immutable flags (e.g., startup scripts like /etc/rc, /etc/rc.local, /etc/rc.d, and others that may be less obvious like /etc/sysctl.conf).  (There is no presecure group defined in the Linux or macOS example configs.) For BSD and macOS, you can choose to make user immutable or system immutable your default, but you can also use groups to explicitly call out a set of files as the opposite of your default, with the group names schg and uchg (and similarly with sappnd and uappnd groups). These groups are specially handled so that they can be specified in combination with another group name (e.g., etc:uchg or acct-logs:uappnd) to identify the files and directories that are in both.

Since it is possible to enable the schg flag in Secure Mode but not to disable it, by default neither syslock nor sysunlock will touch those files while in Secure Mode (sysunlock can't unlock them), but the -f (force) option to syslock will lock them. A group can have both a system and a user subset of files regardless of what default is configured. For example, if your default is schg but you want some commonly modified files in /etc to be uchg, you could put them in that group along with the etc  group. Then, if you unlocked files in Insecure Mode with -g etc, both the schg and uchg files in the etc group would be unlocked, while if you used sysunlock -g etc in Secure Mode, only the uchg files would be unlocked. If you were in Insecure Mode but only wanted to lock the uchg files in the etc group, you can use -g etc:uchg.

The intention of groups is to provide a mechanism for unlocking specifically what is needed to perform a specific task. The default and sample config files supplied with the tool include groups for other tools that will be covered in this series of blog posts, including reportnew (blog post June 16), rsync (for rsync-tools, blog post June 9), and sigtree (blog post June 19), as well as for process accounting log files (to make them append-only), mail servers, web servers, and DNS servers (the latter two in the BSD default config only). Log files and DNS servers provide two examples for where you'd want to use different flag-specific subgroups. For log files, you want the active log file to be append-only, but you want the rotated and archived log files to be immutable; you also want them to use user rather than system flags if the logs are subject to rotation via newsyslog or other log rotation mechanism. Thus the sample configs put the live process accounting log in the uappnd group and the rotated process accounting logs in the uchg group. The rotation process needs to unlock both of those subgroups before rotation, and lock them again after rotation. For DNS configuration files, you might want to lock your zone files with schg if they don't change frequently and aren't changed by any automation, but if you use automated DNSSEC signing, you'd lock the signed zone files with uchg so that they can be unlocked before signing and relocked after signing. It's worth noting that logging to a separate machine is a much better security control than using uappnd (or Linux +a) flags on log files--but I do both.

Overlapping Groups

Note that it is possible to define groups that overlap with each other in various ways, and while this is generally acceptable and unlikely to cause any issues on Linux or if only a single type of flag is used on BSD or macOS, it can create issues if overlapping groups have a mix of flag types such that multiple flags get set on any files or directories. The default and sample configs do not contain any group definitions that create this problem, and syslock will generate warnings if it detects cases of conflicting flag types in the configuration, but will not detect all possible cases (e.g., where symlinks are involved).

OpenBSD-specific: KARL Features

There is also an implicit group of files identified in the config file by a leading "!" prefix character, which designates system files that are part of OpenBSD's Kernel Address Randomized Link (KARL) feature, where the kernel and key binaries (currently libc, libcrypto, ld.so, sshd, sshd-session, sshd-auth, and ssh-agent -- the list may increase between releases) are relinked in random order at reboot.  These files are locked or unlocked by using a -s (for system) option. (This was perhaps a poor choice of option letter and name, as it is distinct from the "system" group which is intended to capture key operating system files and what needs to be unlocked for upgrading and patching, though I typically unlock everything before upgrading or patching. This is just to ensure that nothing the system needs to install is blocked, which can leave a system in an inconsistent and not fully operable state, but the "system" group should actually work for this purpose.) Since I use system immutable flags as my default on most systems, and on the presecure group on all systems, my practice for patches and upgrades is to shut the system down (enter Insecure Mode), unlock everything (sysunlock with no arguments), perform the upgrade (with an extra step) or patch, then lock everything (syslock with no arguments), then unlock what's needed for Karl (sysunlock -s), and then exit, which starts up system daemons and networking, and returns to Secure Mode, without rebooting the kernel or resetting the system uptime counter. In my rc.securelevel file I have the following line:

echo -n ' running syslock'; (/bin/sleep 10; /usr/local/bin/syslock -swf) &

The -s and -f options have already been explained, but -w means to wait for KARL relinking to complete before locking, so as not to prevent that process from occurring. Typically, I do not unlock with -s for most instances when I shut down to make changes, and so I do prevent KARL relinking in those cases, and it doesn't lead to any system inconsistencies--it just preserves the previously established link ordering.

The extra step I mentioned above for an upgrade is that I comment out the line in rc.securelevel that re-locks the system because at the completion of a sysupgrade process there will be a reboot after which I will want to make many additional file changes using sysmerge to update system configuration files, update packages I have installed, remove unused binaries and old libraries, and so forth.  I then remove the comment and reboot again. (Also, if a patch with syspatch rebuilds the kernel, I'll reboot with the new kernel after the return to Secure Mode and the KARL process completes.)

One side note is a little trick I use on my OpenBSD firewall to avoid unnecessary disruption to traffic when doing patching and software updates that require unlocking system immutable flags. The trick is that shutting down to Insecure Mode doesn't impact traffic routing or packet filtering, but running netstart when you exit back to Secure Mode does. (Disruption is unavoidable if it's a kernel patch that requires a full reboot.) The trick is that I've modified my /etc/rc (something I generally avoid doing for ease of upgrades, so I keep the original) by adding one line before and two lines after the call to netstart:

if [ ! -f /var/run/no_netstart_resume ]; then

    sh /etc/netstart

    echo "bypassing netstart"

    /bin/rm -f /var/run/no_netstart_resume

With this in place, I touch /var/run/no_netstart_resume before I exit, and the system restarts with no network disruption.

Audit Feature

Both syslock and sysunlock have an audit (-a) option, which will report which files are currently not in the expected state. That is, syslock -a will tell you all the files which should be locked per the config but are unlocked, and sysunlock -a will tell you all the files which should be unlocked per the config but are locked. This can also be applied to any specific group with -g <groupname>. If used on a BSD system in Secure Mode, the audit can be restricted to what would actually be changed at the current security level using the -o (operational restrictions) option. A -q (quiet) option will suppress all output and just return 0 for success or 1 (error) for failure; this was created to allow a check to see if all files in a particular group are unlocked for an installation to occur for my install.pl tool (to be covered in a June 12 blog post). With -q, the audit will finish and return at the first discrepancy found.

Path Prefixes

The configuration file syntax allows listed paths to use three other prefixes in addition to the OpenBSD-specific "!" referenced above. These prefixes, which only have effect on directory paths (and generate a warning but are otherwise ignored on other file types) are:

+  Do not recurse through subdirectories.  (No prefix, the default, means lock the directory and every thing in it, recursively.)

-  Do not lock the directory itself, just its contents.

=  Lock the directory and its file contents, but not subdirectories.

These cannot be used in combination (though some combinations, like =-, might conceivably be useful), and while I initially made use of the first two path prefixes in my configs, they ended up not being particularly useful with the patterns of groups I developed. I recently added the = path prefix to address a Linux-specific case.

Linux-specific: Bootloader Considerations

On many Linux systems (notably on Proxmox which uses Debian), the grub bootloader rewrites files in /boot/grub, (specifically grub.cfg and grubenv) on reboots. If those files are immutable, the reboot will fail to update those files. The default Linux config creates a grub group for /boot/grub and its contents, and uses the = path prefix on a separate group for /boot and its other subdirectories. The grub group can then be unlocked before a reboot, while the broader group is unlocked for kernel updates, and both locked again afterward. The grub group unlocks slightly more than strictly necessary (everything in /boot/grub rather than the directory and the two specific files), but handles the common case cleanly.

Use of Large Language Models (LLMs)

I've made use of LLMs, initially for performing security assessments with suggested improvements that I'd implement selectively and by hand, but subsequently for working out the details of design for prospective changes, writing code, and identifying the causes of bugs. Each of these capabilities has significantly improved since the initial use for code assessments in the summer of 2025. Specific capabilities implemented by Claude include adding append-only flag support, improving error messages and warnings, code refactoring, enhancing the audit features and adding the -o option, and drafting the Github README. Most recently, while I was in the process of writing this blog post, I revisited an issue I had run into involving overlapping groups with differing flags, and thought that a new path prefix that ignored directories might be a good way to handle it. In addition to that specific use case, for which Claude suggested the use of = (to mean "lock the directory and files at the same level"), I asked whether any combinations of existing path prefixes might be useful or if any other new prefixes might make sense. Claude suggested that -= or =- might be meaningful and useful (don't lock the directory itself, lock the files in it but not the subdirectories), but I chose not to add the additional complexity without a specific use case. In the process I identified a bug in how path prefixes were being handled in previous Claude-generated code related to perl taint handling, and Claude fixed it along with the implementation of =.

I've used ChatGPT and Gemini in addition to Claude for security assessments on the code, and they've each identified different issues. While each has found real bugs, my impression is that I've seen more false positives from ChatGPT and Gemini than from Claude, though they've also found real bugs in code written by Claude.

I've found LLMs quite useful for security auditing, refactoring, and error message improvements, as well as writing new capabilities that are relatively straightforward. In the case of implementing append-only flags, it took multiple design, build, and test cycles to get the code to production quality. Overall, forcing myself to explain design choices clearly (and sometimes reconsider them) in the process of using LLMs has been extremely valuable, and has also been a benefit of writing this blog post. I used Claude to identify key topics that should be mentioned in this series, but did not rely on Claude to outline or write the post. Claude and a human editor reviewed this post (and the opening overview post) identify typos and suggest edits.

I'll include a section like this in each post in the series.

Getting Started and Further Reading

The sample config files supplied with syslock cover OpenBSD, Linux (with some Proxmox specifics in comments), and macOS, and provide a reasonable starting point. As noted above, I recommend starting with uchg groups on BSD and macOS, which gives you immediate error prevention value and a chance to discover what needs to be unlocked for your specific regular workflows before committing to more restrictive schg flags. For Linux, there's only one kind of immutable attribute and it's changeable without a shutdown, so you can start immediately with the sample config.

As noted above, the syslock audit feature is integrated with my install.pl tool and syslock groups more generally are a key feature of both that tool and its distribute.pl counterpart, both of which will be covered in a blog post on June 12. The wrap-up post on June 23 will show how all of these tools fit together into a coherent security architecture.

syslock is available on my website and on Github, along with sample configs for OpenBSD, Linux, and macOS.

25 years of OpenBSD security tools

I've been using and administering OpenBSD systems since 1999 (OpenBSD 2.5). During that time, I've written numerous scripts to make things easier, more automated, or more secure, or sometimes just to improve my understanding of how things work. When I started managing my home systems, I ran several Internet-exposed services on my home network (DNS, mail, web, SSH). I used djbdns, qmail, and Apache httpd at the start before switching to nsd/unbound for DNS and postfix for mail, and finally to OpenSMTPD for mail. When I got tired of excessive inbound traffic I moved my authoritative DNS to a provider while keeping an internal zone and resolvers, set up two cloud servers for mail and my public webserver. My home network became a hardened, minimal-exposure architecture that only allows Wireguard from expected sources and mail (after mutual TLS authentication with certificates) while continuing to run internal services.

Over the years I've made a number of these scripts available via my website and on Github once they were sufficiently mature, configurable, and with reasonable defaults in supplied sample configuration files. A set of them fit together in a coherent framework that I thought I'd write about in a series of blog posts for those who might be interested in either using them or learning from them. All are under BSD licenses which require no permission from me to use or rework into something else -- my intent is not to obtain a new user base that I need to support, though I'm happy to consider suggested enhancements.

The tools I will cover in these posts, while written for OpenBSD originally, also work on Linux and macOS, and likely on other BSDs. I use most on Kali and Proxmox (Debian Linux), as well as a few on macOS (which has roots in FreeBSD). All are written in perl and use OpenBSD-specific features when available (such as pledge and unveil, covered in an earlier blog post). Several use privilege separation to run as much functionality as possible as a non-privileged user.

The earliest tools, written initially between 1999 and 2004, were intended to provide log monitoring, file integrity monitoring, and to manage file system flags for making files immutable, a feature I hadn't seen widely used until macOS introduced a variant in 2015. Even among OpenBSD users, immutable flags are not commonly used and I recall early warnings that they were difficult to use because many files would have to be made immutable in order to avoid somewhat trivial bypasses.

The later-developed tools in the set I will cover, written in 2022-2024, are for file and software distribution across hosts and providing a perl interface to OpenBSD's signify cryptographic signing and verification tool for use by most of the tools in the set.

I'll note in my discussion how I've used large language models (LLMs) for security assessment, bug fixing, and enhancement on these tools, and where it has been useful and some of the obstacles I've run into along the way.  I'll also describe how they work together in various ways, with a comprehensive view in the final wrap-up. Here's the planned schedule:

* June 2: this overview post

* June 5: syslock and sysunlock (initially written 2004, inspired by a much simpler tool called syslock by George Shaffer).

These tools allow you to define and manage groups of files which use BSD system and user immutable and append-only flags or Linux immutable or append-only flags. The use cases range from genuine security enhancement to error prevention.

* June 9: rsync-tools (rsync-client.pl/rsync-server.pl, initially written 2003; rrsync, a 2022 fork of the version initially written by Joe Smith in 2004 and modified by Wayne Davison, the primary developer of rsync; rsync-altroot.pl initially written 2002).

This is a collection of tools used with rsync to define sets of files to be kept synchronized between hosts, perform backups, and to place restrictions on rsync using various mechanisms.

* June 12: distribute.pl/install.pl (originally written 2022) and some extras (including Signify.pm, originally written 2024)

Two tools for distributing content (plain files, config files, certificates, and signed OpenBSD-style packages) to multiple hosts, verifying signatures and installing on the remote hosts. These are not limited to use on OpenBSD, as the install tool will work on Linux and macOS as well--so long as the content being distributed is usable on those operating systems, which includes architecture/OS-independent OpenBSD-style packages such as perl scripts--like all of the scripts in this blog series, which are available as signed OpenBSD-style packages. These are packaged with some extras (gendoas.pl with distribute.pl and pkg_info.pl with install.pl) and this blog post will also cover Signify.pm, a perl interface to OpenBSD's signify.

* June 16: reportnew (originally written 1999)

A log monitoring tool, inspired by swatch but written to support djbtools' cyclog and multilog format logs, Linux journal files, as well as BSD, Linux, and macOS process accounting logs, the latter of which may be a feature unique to this tool (I've not seen it in any other log monitoring tool).

* June 19: sigtree.pl (originally written 2000)

A file integrity monitoring tool, inspired by the original tripwire.

* June 23: Wrap-up post

Each post will discuss the origins of the tool(s), the use cases and problems solved, the security model, the limitations, and, in some cases, what might be added in the future.  As each post is published, links will be added above.

If you want to skip ahead and look into these tools further, all are available on Github and on my website.

Tips on using OpenBSD's pledge and unveil in perl scripts

 OpenBSD 5.9 (current as of this post is 7.5) introduced the "pledge" system call and 6.4 introduced the "unveil" system call, which together provide a means of more granular control of system access by processes running on the system to enforce least privilege.  When a program calls "pledge", it provides a list of categories of system calls (called "promises") that it is planning to make during the life of the running process (children have to make their own pledges and are not restricted), and attempts to make calls outside of those areas will cause the call to be blocked and the process to be killed. Additional calls to pledge cannot add new categories but it can remove them, so access can become more restrictive but not less restrictive.

  "Unveil," by contrast, selectively exposes parts of the file system, by file path, with specific access, and the rest of the file system is correspondingly "veiled" or blocked from access. Successive calls to unveil can expand or override previous ones, expanding access to the file system, adding write and create permissions where there was previously read only, but only until unveil is called with no arguments, which locks the current state in place. Further attempts to call unveil after that result in a violation.

Violations of pledges or attempts to access file paths that are not unveiled show up in process accounting logs for the process with the process flags "P" or "U", respectively.  (My "reportnew" log monitoring script knows how to monitor process accounting logs and can be easily set up to report on such violations.)

Perl scripts on OpenBSD can also use pledge and unveil, with two modules provided in the base operating system, "OpenBSD::Pledge" and "OpenBSD::Unveil".  I've been adding this functionality to several of my commonly used scripts and have learned a few tips that I'd like to share.

Pledge:

* Check your call to pledge for errors.  If you typo the name of a promise (category of system calls), or you provide pledge with a string of comma separated promises instead of an array or list, it will fail and nothing will be pledged.

* If you don't have any idea what promises are required, just use "error".  With the error promise, instead of blocking the system call and killing the process, the result is logged to /var/log/messages and you can see what promises are required.

* The "stdio" promise is always included with OpenBSD::Pledge, so you don't need to list it.

* The "unveil" promise is required if you intend to use OpenBSD::Unveil.

* Calls to exec or system require the "proc" and "exec" promises; the new processes created as a result are not restricted and need to make their own use of pledge and unveil.  (Note: this means that if you are calling a system command that writes to a file, but your script doesn't otherwise write to files, you do not need to pledge the "wpath" promise in your script.)

* If you otherwise fork a child process (e.g., explicitly using "fork" or Parallel::ForkManager or implicitly forking a child process using "open" to read from or write to a command), the promises pledged by the parent process are carried over to the child, which can then restrict them further. (Hat tip to Bryan Steele, who pointed this out on Bluesky without specifically referring to the Perl context.)

* If you use the DBI perl module with mariadb and are accessing a database through a named pipe on the same server, you'll need to pledge the "unix", "inet", and "prot_exec" promises. (This works even from a chroot jail if the named pipe or socket is a hard link from the jail.)

* This isn't a tip, but an observation: if you promise "proc" but not "exec," your system call will fail but your process will not be killed and the script will continue running.

Unveil:

* If you make use of other perl modules in your code with "use", they are loaded prior to your call to unveil and so you don't need to unveil directories like /usr/libdata/perl5 in order to use them. The exception is perl modules that include compiled shared objects (".so"), or which use "require" on other modules (loading them at runtime), in which case you do need unveil such directories, but only with "r" permission.

* If you use the DBI perl module with mariadb, you will need to unveil /var/run/mysql with "rw" and /usr/lib and /usr/local/lib with "rx".

* If you use calls to "system" or "open" which use pipes, globs, or file redirection, you need to unveil "/bin/sh" with "x" permission. You may be able to rewrite your code to avoid the requirement--can you call "system" with a command name and list of arguments rather than a string, and do any processing you need in your program instead of with the shell?

* If you use calls to "system" to execute system commands, you need to unveil them with "x" permission but in most cases you don't need to include "r".

* It is often much easier to unveil a directory rather than individual files; if you plan to check for the existence of a file and then create it if it doesn't exist, you need "rwc" on the containing directory.

* One of the biggest challenges sometimes is to find the source of an unveil violation; unveiling "/" with various permissions to see if it goes away, and then removing that and testing individual directories under the root directory in trial and error can help find things. That's how I first found the need to unveil "/bin/sh". 

Finally, if you are writing perl modules it's helpful to document which promises need to be pledged and files and directories need to be unveiled in the calling scripts in order for them to function. It would be inappropriate to pledge or unveil within the module except in a context like a forked child process. I've done this with my Signify.pm wrapper for the OpenBSD "signify" command for signing and verifying files with detached signatures or gzip archives with embedded signatures in the gzip header comments.

If you've made use of pledge and unveil--in perl scripts or otherwise--what lessons have you learned?

[UPDATE: 8 February 2026]: Note that OpenBSD::Pledge still does not support execpromises, so you cannot place pledge's execpromises restrictions on anything executed with "system" or "exec."]

[UPDATE 1 April 2026]: The "tmppath" promise is being removed in favor of just using "unveil" for any permissions limitations on files in /tmp. Note that if permission is granted to a symbolic link, it also applies to the target of the link, so granting "rwc" to /tmp opens potential bypass of unveil restrictions via symlinks in /tmp.

[UPDATE 14 May 2026]: Even though neither pledge nor unveil are inherited across system calls, it is possible for an error in a perl script to cause unveil errors in another process invoked via system call -- not because of its own unveil environment, but by messing with low-numbered file descriptors (e.g., by messing with STDOUT or STDERR in an inappropriate way), which I found by accident while modifying a script to use direct calls to ping instead of using Net::Ping, and suddenly saw unveil violations from ping processes run by the _ping user. Specifically, such an error can cause libc to attempt a path-resolution write inside ping, which fails because ping unveils / with r for read-only permission.

[UPDATE: 6 June 2026]: If you pledge 'fattr' but not 'chown' in a script that uses Archive::Tar to extract files, it will generate an error when you extract a file with a group that root is not a member of, as this is a restriction even on root for 'fattr' (but not 'chown'). This was mystifying for a while because the chown error (which doesn't include the "Operation not permitted" message) came from Archive::Tar but everything still ended up in the correct state because the group ID was inherited from the containing directory. It was further complicated since I was locking and unlocking immutable flags (which was working just fine!) and it first seemed like the most likely culprit.

Google Bard on file system immutability flags in OpenBSD

 Unlike ChatGPT, Bard pretty much nailed it on the initial prompt. (However, Bard refused to answer my questions about the Mars Effect like the ones I gave to ChatGPT, even when reworded to be clear I was asking for descriptions of what the effect is and what explanations are offered rather than asking it to identify causes itself.)

It does lose points for grammar in the second sentence, though!

write a few paragraphs explaining immutable flags in OpenBSD including the commands for setting them and the difference between system and user immutable flags

Sure. Here is a few paragraphs explaining immutable flags in OpenBSD, including the commands for setting them and the difference between system and user immutable flags.

Immutable Flags

Immutable flags are a security feature in OpenBSD that prevent files from being modified, moved, or deleted. They can be set on any file by the superuser, or on files owned by the current user with the chflags command.

To set the immutable flag on a file, use the following command:

chflags schg filename

To unset the immutable flag, use the following command:

chflags nouchg filename

System and User Immutable Flags

There are two types of immutable flags: system immutable and user immutable. System immutable flags are set by the superuser and can only be cleared by the superuser at security level 0 or -1. User immutable flags are set by the owner of the file and can be cleared by the owner of the file at any time.

Commands for Setting Immutable Flags

The following commands can be used to set immutable flags:

• chflags schg filename: Sets the system immutable flag on filename.
• chflags uappnd filename: Sets the user append-only flag on filename.

Difference Between System and User Immutable Flags

The main difference between system and user immutable flags is that system immutable flags can only be cleared by the superuser at security level 0 or -1, while user immutable flags can be cleared by the owner of the file at any time.

Example

The following example shows how to set the immutable flag on a file and then clear it:

# Set the immutable flag on the file /etc/passwd
chflags schg /etc/passwd

# Clear the immutable flag on the file /etc/passwd
chflags nouchg /etc/passwd

Benefits of Using Immutable Flags

Immutable flags can be used to improve the security of your system by preventing unauthorized changes to critical files. For example, you could use immutable flags to protect system configuration files, such as /etc/passwd and /etc/shadow, from being modified by attackers.

Disadvantages of Using Immutable Flags

There are a few disadvantages to using immutable flags. First, they can make it difficult to troubleshoot problems with your system. If a file that is set to immutable is corrupted, you will not be able to modify it to fix the problem. Second, immutable flags can slow down your system performance, especially if a large number of files are set to immutable.

Conclusion

Immutable flags are a powerful security feature that can be used to protect your system from unauthorized changes. However, it is important to weigh the benefits and disadvantages of using immutable flags before deciding whether or not to use them on your system.

Some nonsense from ChatGPT about file system immutability flags in OpenBSD

 I was thinking about writing a blog post about file system immutability flags in OpenBSD, and thought I'd ask ChatGPT to write it for me. It appears to have gleaned its content from OpenBSD man pages, but it is highly misleading and inaccurate, misrepresenting the key distinction between system and user immutability flags.

What it got right is that only the superuser (root) can set and unset the system immutability flag (schg/noschg) while users (if they own the file) can set and unset the user immutability flag (uchg/nouchg). But either flag can be set or unset on any kind of file.

The user immutability flag can be unset (nouchg) by either the superuser or the owner of the file regardless of the system security run level (-1 to 2), but the system immutability flag can only be unset (noschg) by the superuser when the system security run level is 0 (insecure mode) or -1 (permanently insecure mode).

During boot, the system normally starts in single-user mode at security run level 0. The init process raises the run level to 1 (secure mode) when booting to multi-user mode, and to 2 (highly secure mode) if the rc.securelevel config file so specifies. The system secure level can also be raised if the superuser changes it with the sysctl command, setting the kern.securelevel variable. The permanently insecure mode (-1) can be set with sysctl only while in insecure mode, which will cause the init process to not raise the security level to 1 during boot to multi-user mode.

The man page for securelevel gives an overview of the restrictions at each security level. I run my systems at secure level 2, highly secure mode. Secure mode (secure level = 1) and above prevent lowering the secure level except by the init process (when the system is shutdown back to single user mode), do not allow /dev/mem or /dev/kmem to be opened, make raw disk devices read only, and disallow unsetting of system immutability flags (or unsetting the related system append-only flags, sappnd, using nosappnd), along with a few other restrictions. Highly secure mode (securelevel = 2) also prevents setting the time backward or close to overflow, and prevents changing host firewall rules (pf filter and NAT rules), though IP addresses may still be added or removed from tables. The most significant difference between the system and user immutability flags is not what files they are set on, but that the user immutability flags can be turned off at any time by the file owner or superuser, while the system immutability flags can only be changed while in insecure modes, i.e., after a shutdown from multi-user operation.

I use system file immutability flags on most configuration files, system binaries, and system libraries as an obstacle to an attacker in the event of a compromise and escalation of privileges. If an attacker has console access to a system, they can bypass these flags by shutting down the system and unsetting the flags. If I left certain config files without file system immutability flags set, an attacker could modify one of those config files that leads to execution during system startup to insert commands to unset the flags, and then reboot the system so that those commands execute while the system is in insecure mode.

These flags can get in the way and cause problems without an efficient way to manage them, as they need to be unset in order to do system patches and updates--it's bad news if you try to do a system upgrade while many files are set with immutability flags. They also block features like OpenBSD's kernel address randomized link (KARL) and library reordering address space layout randomization (ASLR) features, which relink the kernel and certain key libraries on reboot to make blind return-oriented programming (BROP) attacks more difficult. I use scripts called syslock/sysunlock to manage the immutability flags. These scripts have a config file (kept immutable, of course) that specifies which files and directory contents to set immutable, and defines groups so that I can unlock what I need at a particular time with a single command. When I want to allow KARL and library ASLR to occur, I do a sysunlock -s (unlock system files) while in single user mode and then boot. I've also defined groups for things like "everything in /etc", "everything related to DNS", etc.

My rc.securelevel config file contains a command to sleep for a period of time, then do a syslock -s after raising the system security level, giving the system time to do the relinking, so I don't need to issue a command to re-lock those files. This is a process I also follow when installing kernel patches, as syspatch fails to do kernel relinking when in single-user mode. That is, I boot to single-user mode, unlock the system files with "sysunlock -s", then exit back to multi-user mode and the KARL process (/usr/libexec/reorder_kernel) and library ASLR (part of /etc/rc) run without error.

syslock/sysunlock may be found at https://www.discord.org/lippard/software/ and on Github at https://github.com/lippard661/syslock

Anyway, after that long introduction, here's what ChatGPT had to say about all this; my prompts are in red:

write a few paragraphs explaining immutable flags in OpenBSD including the commands for setting them and the difference between system and user immutable flags

In OpenBSD, immutable flags are used to protect files from being modified or deleted. These flags can be set on both system and user files, and are useful for preventing accidental or malicious changes to important system files.

To set an immutable flag on a file, the "chflags" command can be used. The command takes the form "chflags immutable [filename]" and can be used to set the flag on a single file or multiple files at once. To remove the immutable flag, the command "chflags noschg [filename]" can be used.

The difference between system and user immutable flags is the level of protection they provide. System immutable flags are set on system files and are only modifiable by the superuser. User immutable flags are set on user files and can be modified by the owner of the file. This allows users to protect their own files, while also allowing the system administrator to protect important system files.

It's important to note that while immutable flags provide a level of protection, they should not be considered as a replacement for proper file permissions and security practices. It's still important to use appropriate permissions to control access to files and to keep your system updated to protect against known vulnerabilities.

Can I set the user immutable flag on a system file or the system immutable flag on a user file?

In OpenBSD, the system immutable flag can only be set on system files, and the user immutable flag can only be set on user files. Attempting to set a user immutable flag on a system file or a system immutable flag on a user file will result in an error.

This is because the system immutable flag is intended for use on system files that should not be modified by anyone other than the superuser. Setting this flag on a user file would not provide any additional security, as the user already has full control over their own files. Similarly, the user immutable flag is intended for use on user files that should not be modified by the owner. Setting this flag on a system file would not provide any additional security, as the superuser already has the ability to modify system files.

It's important to use the appropriate immutable flag for the type of file you are working with to ensure that the file is protected as intended.

[UPDATE, 22 Jan 2023:]

That is incorrect. The system immutable flag can be set on user files and the user immutable flag can be set on system files. The difference is not what files they can be used on, but that the system immutable flag can only be unset by the superuser when the system is in insecure mode, while the user immutable flag can be unset by the file owner or superuser at any system security run level.

I apologize, you are correct. The system immutable flag can be set on user files, and the user immutable flag can be set on system files. The difference between the two flags is not the type of file they can be applied to, but rather the level of protection they provide and who can unset them.

The system immutable flag provides a higher level of protection and can only be unset by the superuser when the system is in insecure mode, This means that once the system flag is set, the file cannot be modified or deleted without booting in insecure mode.

On the other hand, the user immutable flag can be unset by the file owner or the superuser at any system security run level. This means that the file can be modified or deleted by the owner of the file or superuser at any time.

Again, I apologize for the confusion in my previous answer. Thank you for bringing this to my attention.

A few thoughts on OpenBSD 5.8

I've been using OpenBSD since way back at release 2.3 in 1998, so I've gone through upgrades that took a fair amount of work due to incompatible changes, like the switch from ipf to pf for host firewalling or the change to ELF binaries. The upgrade from 5.7 to 5.8 was a pretty smooth and easy one, for the most part. The two most painful changes for me were the replacement of sudo with doas and the dropping of support in the rc.conf for the pf_rules variable.  While sudo is still available as a package, I like the idea of reducing attack surface with a simpler program, so I made the switch. The two things I miss most about sudo are the ability to authenticate for a period of time and the ability to have a single config file across a whole set of servers. The former I'm just living with, the latter I've adjusted to by having a single config file that has lines commented out depending on which server it's on. I did have one moment of concern about the quality of doas when it incorrectly reported the line number on which I had a syntax error in the config file--fortunately, this was just a failure to increment the line count on continuation lines (ending with a "\") which is fixed in the -current release.

The removal of the pf_rules variable support from rc.conf was a bigger issue--I used to just put the default pf.conf rules file in place with each release and upgrade, and keep my changes in a pf.conf.local file that was specified in the pf_rules variable. The effect was that from the period after the upgrade until I noticed the change, my systems were using the default rules and thus more exposed than they were supposed to be. This wasn't the first time an incompatible change decreased my level of security--the removal of tcpwrappers support from SSH was another. I used to use a combination of pf rules and hosts.allow as additional layers of protection on my SSH access, and had a set of tools that allowed me to easily add IP addresses to or remove them from my hosts.allow files. This would have been a layer of defense still in place with the loss of my pf rules, had it still been in existence. Fortunately, I also have SSH on a non-standard port and only allow SSH key logins, not user/password logins, and most of my systems can't be reached on any port without first making a VPN connection, which requires two-factor authentication.

A minor annoying change that was made in 5.8 was putting the file /var/unbound/db/root.key into /etc/changelist, so that the file gets checked daily by the security script. The issue with this is that if you are actually using unbound with DNSSEC, this file changes daily, though only in the comments. My "reportnew" log monitoring tool has a feature that allows you to be notified if files that are expected to change on some periodic schedule do not change, and that would be more appropriate than getting daily notifications that yes, the autotrust anchor file has been updated yet again. But what would really be ideal here would be a check that the non-comment components have not changed. (Others have also complained about this.)

A final issue I've run into with OpenBSD 5.8 is not a new issue, but it's one that still hasn't been fixed with pf. That is that pf logs certain traffic (IGMP in particular) when it matches a rule that does not call for logging. This appears to be the same issue that was fixed earlier this year in pfsense, which is derived from an older fork of pf.

How to use Google Authenticator with OpenBSD, OpenSSH, and OpenVPN--and why you might not want to

I thought that Google Authenticator might be a quick and easy two-factor authentication solution for VPN access to my personal network, so I did some Google searches to see if that were so.  I found quite a few sources describing how to set it up with systems that use Linux Pluggable Authentication Modules (PAM), but very little about using it with BSD Authentication on OpenBSD.

The most promising link I came across was to an implementation of Google Authentication for OpenBSD that was last updated in early 2013, based on Google's PAM code, but I couldn't get it to work.  It compiled and installed, and the googleauth code for generating a secret (and a very insecure way of generating a QR code to use to import it into the Google Authenticator application) worked fine, but I couldn't successfully use it for console login, OpenSSH login, or OpenVPN login.

I also found the standard OpenBSD port for openvpn_bsdauth, which compiled, installed, and worked successfully for password authentication by adding these lines to my OpenVPN configuration:

script-security 2
tmp-dir <path to dir writable only by _openvpn user>
auth-user-pass-verify /usr/local/libexec/openvpn_bsdauth via-file

This also requires that the authenticating user be put into the _openvpnusers group.

I was unable to get the via-env method to work, however.

I next tried the standard OpenBSD port of login_oath, which implements the OATH toolkit, which uses the same time-based TOTP protocol that Google Authenticator uses.  This turned out to do the trick.  Once installed, you create a secret key that the server authentication will check against and store it in your home directory (one thing I liked about googleauth is that it stores the shared secret in a system directory to which the user doesn't have access; better still is the suggestion of keeping the secrets on an auth server as totp-cgi does).  The documentation recommends creating the secret (which the user doesn't need to know except for the initial configuration of the Google Authenticator client application) by doing:

openssl rand -hex 20 > ~/.totp-key

I then needed to convert this from hex to base32, which is simple enough to do with the method the documentation recommends, which is using the perl module Convert::Base32 (OpenBSD port p5-Convert-Base32) and a short script like:

#!/usr/bin/perl
use Convert::Base32;
open (FILE, "/home/vpnuser/.totp-key");
$secret = <FILE>;
close (FILE);
$code = pack ('H*', $secret);
print encode_base32($code)."\n";

The resulting code can be manually entered into Google Authenticator.

To use Google Authenticator as a login method, I updated the login class for the user I wanted to use in /etc/login.conf so that its last two lines were:

:auth=-totp,passwd:\
:tc=default:

This allows either Google Authenticator or password authentication at the console, but only Google Authenticator via OpenSSH or OpenVPN as I configured them.  Instead of using "-totp" you can also use "-totp-and-pwd" which requires the entry of both your Google Authenticator code and your password (in that order, with a slash in between them) in order to authenticate.

For OpenSSH, I added the following lines to my sshd_config:

Match User <vpnuser>
     PasswordAuthentication yes
     AuthenticationMethods publickey,password:bsdauth

I don't allow password authentication at all for other users; for this user, an SSH public key must first be used, then Google Authenticator must also be used before a successful login. [Updated 1 Nov 2013 to add:  After a reboot, this ssh config failed with a log message of "fatal: auth2_update_methods_lists: method not in AuthenticationMethods".  Removing the ":bsdauth" made it work again (it works since the "password" authentication method will use BSD Authentication by default), but this looks like an SSH bug.]

So why might you not want to do this?  While Google Authenticator ensures that what is used over the network as a password is better than a typical user-selected password, it effectively stores a shared secret in plaintext at both ends of the connection, which is far less secure than SSH public key authentication.  If the device where Google Authenticator is present gets compromised, that secret is compromised.  And as the above link about totp-cgi points out, if you use Google Authenticator with the same secret across multiple machines, that secret is only as secure as the least secure host it's stored on, and using different secrets for different machines doesn't scale very well with the application.  A password safe with randomly generated passwords, stored in encrypted form, is probably a better solution in most cases. [Updated 2 November 2013: Authy uses the same TOTP mechanism as Google Authenticator, but encrypts the secret key on the client side.  That encryption is really more obfuscation than encryption since the key is based on phone attributes and can potentially be reverse engineered.]

As I've set it up, I'm still relying on SSH public key authentication for SSH logins, and on certificate authentication for VPN logins, in addition to Google Authenticator.  For the case of logging into my VPN from my laptop and having Google Authenticator on a separate mobile device, it does seem to be a security improvement (though I welcome anyone to show me that the gains are illusory).

UPDATE (July 31, 2019): Note that you should make the .totp-key file in the user's home directory owned by and only readable by root, or else you're effectively permitting that user to do passwordless doas/sudo, since passworded doas/sudo will use the TOTP mechanism for authentication. That won't stop the user from removing the .totp-key file and replacing it with their own, but at least that action becomes detectible. To prevent removal, on OpenBSD you can set the file to be immutable (schg flag) and run at securelevel=2. But a better solution would really be to put those secrets somewhere outside of the individual user's home directory.

UPDATE (October 22, 2019): The OpenVPN authentication with 2FA is broken in OpenBSD 6.6, it now leads to user/password authentication failures. Not sure why yet.

UPDATE (October 22, 2019 #2): Looks like it may have been user error, it works now, though I did update my _openvpnusers group to the new number (811) from the old one (596), but the number itself shouldn't be hardcoded in openvpn_bsdauth, so that shouldn't have had an impact.

UPDATE (30 October 2022): Also see Solene Rapenne's blog post on this same topic.

UPDATE (18 June 2024): Note that Authy (and probably any other authenticator app) treats manual secrets entry as case-sensitive even though base32 is not, and secrets must be entered in uppercase.

UPDATE (9 September 2025): I recently switched from VMware vSphere ESXi to Proxmox and implemented 2FA for SSH using the same codes that are used for 2FA for the web interface. There are a number of guides online that I drew from (such as this one), but none of them gave me quite what I was used to--most of them allowed password + 2FA and I wanted to force SSH keys + 2FA.  I installed libpam-google-authenticator, added the line "auth required pam_google_authenticator.so nullok" and commented out the "@include common-auth" line that allows passwords. In sshd_config, I added "PasswordAuthentication no", "PermitRootLogin prohibit-password", and "KbdInteractiveAuthentication yes". I added an "AllowUsers" list with root, my account, and my rsync account, and then did a "Match User" for my account with "AuthenticationMethods publickey,keyboard-interactive" to require both. (No 2FA requirement for root SSH as that would impact node-to-node authentication for clustering, but it's forced to use SSH keys and can be restricted by IP. Too bad it doesn't use a non-root account, though.)

UPDATE (31 October 2025): To make my Proxmox system consistent with my OpenBSD boxes, I switched from libpam-google-authenticator to libpam-oath and also made it required authentication for doas. I'm also using a centralized, system administrator managed, visible-only-to-root config file for TOTP keys.