user@argobox:~/journal/2025-03-17-the-pool-that-refused-to-import
$ cat entry.md

The Pool That Refused to Import

○ NOT REVIEWED

The Pool That Refused to Import

Date: 2025-03-17 Duration: About 30 minutes Issue: ZFS pool import failing on boot after Proxmox reinstall Root Cause: Ghost pool from previous installation blocking import


The Situation

Fresh Proxmox install. Installed over an old Proxmox system — same machine, same drives, clean slate.

Boot up. Error message on the console:

Failed to start Import ZFS pools by device scanning

Every single boot. Red text in the boot log.


The First Investigation

Logged in and checked for pools:

zpool list
no pools available

No pools. But the error says it’s failing to import pools. Something is finding a pool somewhere.

Checked the systemd services:

systemctl | grep zfs-import
● zfs-import-scan.service loaded failed failed Import ZFS pools by device scanning
  zfs-import.target        loaded active active ZFS pool import target

zfs-import-scan.service is in a failed state. The scan found something it couldn’t handle.


The Ghost Pool

Dug into the service status:

systemctl status zfs-import-scan.service
Process: 847 ExecStart=/sbin/zpool import -aN -d /dev/disk/by-id (code=exited, status=1/FAILURE)

zpool[847]: cannot import 'tank': pool was previously in use from another system.
zpool[847]: Last accessed by Icarus-Orchestrator (hostid=7dcb3aa3) at Mon Mar 17 02:11:29 2025
zpool[847]: The pool can be imported, use 'zpool import -f' to import the pool.

There it is. A pool called tank from the previous Proxmox installation. ZFS found it on the drives, but the hostid didn’t match (new install, new hostid). It refused to auto-import a “foreign” pool.


The Decision

Two options:

Option 1: Force import the old pool

zpool import -f tank

Option 2: Destroy it and start fresh

Since this was a clean reinstall and I wanted fresh pools anyway, went with option 2.


The Cleanup

First, disabled the failing services:

systemctl disable zfs-import-scan.service
systemctl disable zfs-import-cache.service

Force imported the old pool so we could destroy it:

zpool import -f tank
zpool list
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
tank  10.9T   200M  10.9T        -         -     0%     0%  1.00x  ONLINE  -

Destroyed the old pool:

zpool destroy tank

Wiped the ZFS partition labels from the drives:

wipefs -a /dev/sdb
wipefs -a /dev/sdc
wipefs -a /dev/sdd
wipefs -a /dev/sde

Creating New Pools

Created a RAID10 pool for VMs and containers:

zpool create -o ashift=12 tank-storage mirror /dev/sdb /dev/sdc mirror /dev/sdd /dev/sde

Set optimal properties:

zfs set compression=lz4 tank-storage
zfs set atime=off tank-storage

Updated the ZFS cache so it imports cleanly:

zpool set cachefile=/etc/zfs/zpool.cache tank-storage
update-initramfs -k all -u

Re-enabled the cache import service (but not scan):

systemctl enable zfs-import-cache.service

Rebooted. No more errors.


What I Learned

ZFS remembers pools across reinstalls. The pool metadata lives on the drives, not in the OS. A new Proxmox install will still find the old pools during device scanning.

Hostid mismatch blocks auto-import. ZFS uses the system’s hostid to track which machine last accessed the pool. New install = new hostid = refusal to import without -f.

zpool list shows imported pools, not existing ones. To see all pools on the system (imported or not), use:

zpool import

Create pools via CLI, not the web UI. The Proxmox web UI creates extra service files that can cause duplicate import attempts. CLI is cleaner.

Use cache import, not scan import. zfs-import-cache.service imports pools from a known list. zfs-import-scan.service scans all devices. Cache is faster and more predictable.


Quick Reference

# See all pools (imported or not)
zpool import

# Force import a pool from another system
zpool import -f poolname

# Destroy a pool completely
zpool destroy poolname

# Update the import cache
zpool set cachefile=/etc/zfs/zpool.cache poolname
update-initramfs -k all -u

# Wipe ZFS labels from a drive
wipefs -a /dev/sdX

“Failed to start Import ZFS pool.” No pools listed. But there was a pool — hiding in the drives, waiting to cause trouble.