If you run pfSense on Netgate appliances, this Tech Note is for you. pfSense on Netgate appliances (locally; not cloud-based) has been my firewall of choice for nearly 15 years. The devices are rock solid with the security of FreeBSD to boot. However, the devices use ZFS, which can be confusing at times.
I have a Netgate 2100 at one of my remote sites, serving as a firewall/Wireguard client connection back to the main office. The 2100 is ideal in this role, providing all the power you need for a remote client site with 4 GB RAM, and an 8 GB MMC disk. While Netgate creates separate ZFS filesystems for /var (and several subdirectories), /tmp, and /home, in reality these are all on the same physical disk, which you'll note when viewing the output from the df command:
Filesystem Size Used Avail Capacity Mounted on
pfSense/ROOT/default_20250825095103 4.7G 1.0G 3.6G 22% /
devfs 1.0K 0B 1.0K 0% /dev
pfSense/var 3.6G 3.2M 3.6G 0% /var
pfSense/tmp 3.6G 228K 3.6G 0% /tmp
pfSense/home 3.6G 96K 3.6G 0% /home
pfSense/var/log 3.6G 3.1M 3.6G 0% /var/log
pfSense/var/cache 3.6G 96K 3.6G 0% /var/cache
pfSense/var/db 3.6G 2.5M 3.6G 0% /var/db
pfSense/var/tmp 3.6G 104K 3.6G 0% /var/tmp
pfSense/ROOT/default_20250825095103/cf 3.6G 1.3M 3.6G 0% /cf
pfSense/ROOT/default_20250825095103/var_cache_pkg 3.9G 321M 3.6G 8% /var/cache/pkg
pfSense/ROOT/default_20250825095103/var_db_pkg 3.6G 7.5M 3.6G 0% /var/db/pkg
tmpfs 4.0M 140K 3.9M 3% /var/run
Wait, something's fishy here! I'm supposed to have 8 GB of disk space. Why do I only have 4.7?
pfSense/ROOT/default_20250825095103 4.7G 1.0G 3.6G 22% /
Prior to the upgrade, the situation was even worse, with df reporting only 1.8 GB of disk space total! What gives?
Smartly, Netgate appliances running their pfSense+ software implement what are called "boot environments" - ZFS snapshots taken before each upgrade. You can view the saved boot environments from the "System > Boot Environments" menu in the pfSense+ web GUI, however, the space occupied by each is misleading; for example, on the web page, it shows the snapshot from my previous upgrade as occupying 456 kb of space. In order to see the true amount of used space, you will need to view the snapshots from the command line with the command zfs list -t snapshot:
[25.07.1-RELEASE][root@wfmfw004.wwf-mgt.com]/root: zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
pfSense/ROOT/default_20250825095103@2025-08-25-09:51:12-0 308K - 1.04G -
pfSense/ROOT/default_20250825095103@2025-08-25-10:11:12-0 316K - 1.04G -
pfSense/ROOT/default_20250825095103/cf@2025-08-25-09:51:12-0 140K - 6.24M -
pfSense/ROOT/default_20250825095103/cf@2025-08-25-10:11:12-0 140K - 6.24M -
pfSense/ROOT/default_20250825095103/var_cache_pkg@2025-08-25-09:51:12-0 137M - 137M -
pfSense/ROOT/default_20250825095103/var_cache_pkg@2025-08-25-10:11:12-0 88K - 120K -
pfSense/ROOT/default_20250825095103/var_db_pkg@2025-08-25-09:51:12-0 1.02M - 7.45M -
pfSense/ROOT/default_20250825095103/var_db_pkg@2025-08-25-10:11:12-0 1.01M - 7.44M -
Note the REFER column, which shows my previous snapshot, taken on 8/25/2025 at 09:51:12 as occupying 1.04 GB. Prior to cleaning up these snapshots, I had 4 of them from previous upgrades, occupying over 85% of my available space! Even sneakier, Zabbix was none the wiser, as my firewalls weren't reporting 8 GB of total space; as each snapshot was taken, my "total" disk size decreased!
The solution? In the Boot Environments page in the pfSense+ web GUI, delete all boot environments other than the current running environment. After deletion, the df command will immediately reflect the increase in available disk space for the root filesystem. At this point, you can proceed with your upgrade.
Note: for upgrading pfSense+, I would recommend at least 1.5 GB of free space, otherwise, the upgrade will fail with an "out of space" error. After the failure, you will need to manually remove the downloaded packages from /var/cache/pkg or your risk an inaccessible device!
After freeing up the necessary space, I decided to upgrade from the shell via the pfSense-upgrade -d command. Be aware - your firewall will automatically reboot once the upgrade is complete.
For all its quirks, I remain a staunch Netgate-pfSense guy (though the pfSense fork OPNSense is a good solution as well). I want a firewall that works even if the Internet is down and that doesn't require an "always on" cloud connection in order to function. Netgate support has always been extremely knowledgeable and responsive anytime I've needed them, and I've had more than one appliance that lasted a decade+.
If you're a pfSense user and you're in a jam, feel free to reach out and I'd be happy to help if I can - matt@thesoloadmin.com