It seems that ZFS cannot remove special vdevs from any kind of RAIDZ pools as of OpenZFS v2.1.4. This behaviour is believed to be a specification being caused by a limitation of top-level vdev removal, I guess.
Suppose the pool which consists of a RAID-Z1 vdev and a mirrored special vdev exists.
# zpool status ztank
pool: ztank
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
zdata ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0
special
mirror-1 ONLINE 0 0 0
da4 ONLINE 0 0 0
da5 ONLINE 0 0 0
errors: No known data errors
Now, run zpool remove command to remove the special vdev, and an error occurred like below.
# zpool remove ztank mirror-1 cannot remove mirror-1: invalid config; all top-level vdevs must have the same sector size and not be raidz.
I think that removing top-level vdev “mirror-1” all at once is not good, then try to remove each member of the vdev, but no sense.
# zpool detach ztank da5 # zpool remove ztank da4 cannot remove .....
By the way, it is possible to remove slog or L2ARC which is one of top-level vdev kinds from any RAIDZ pools for some time, and besides, non-RAIDZ pools have no limitation to remove top-level vdevs.
I have been preparing to apply a special vdev to my RAIDZ2 pool on real part, but it is a little hard to do it. It was good I tried these beforehand on a VM… L2ARCs and slogs are removal from any RAIDZ pools so special vdevs will become possible to do in a future or else it won't because the special vdev removal have to be strictly evacuated data in the vdev to main vdev?
I will have no choice but to alternatively use p2L2ARC for the time being.