====== ZFS can't remove special vdevs from RAIDZ pools ======

It seems that ZFS cannot remove special vdevs from any kind of RAIDZ pools as of OpenZFS v2.1.4. This behaviour is believed to be a specification being caused by a limitation of top-level vdev removal, I guess.

Suppose the pool which consists of a RAID-Z1 vdev and a mirrored special vdev exists.

<code>
# zpool status ztank
  pool: ztank
 state: ONLINE
config:

        NAME        STATE     READ WRITE CKSUM
        zdata       ONLINE       0     0     0
          raidz1-0  ONLINE       0     0     0
            da1     ONLINE       0     0     0
            da2     ONLINE       0     0     0
            da3     ONLINE       0     0     0
        special
          mirror-1  ONLINE       0     0     0
            da4     ONLINE       0     0     0
            da5     ONLINE       0     0     0

errors: No known data errors
</code>

Now, run ''zpool remove'' command to remove the special vdev, and an error occurred like below.

<code>
# zpool remove ztank mirror-1
cannot remove mirror-1: invalid config; all top-level vdevs must have the same sector size and not be raidz.
</code>

I think that removing top-level vdev "mirror-1" all at once is not good, then try to remove each member of the vdev, but no sense.

<code>
# zpool detach ztank da5
# zpool remove ztank da4
cannot remove .....
</code>

By the way, it is possible to remove slog or L2ARC which is one of top-level vdev kinds from any RAIDZ pools for some time, and besides, non-RAIDZ pools have no limitation to remove top-level vdevs.

I have been preparing to apply a special vdev to my RAIDZ2 pool on real part, but it is a little hard to do it.
It was good I tried these beforehand on a VM...
L2ARCs and slogs are removal from any RAIDZ pools so special vdevs will become possible to do in a future or else it won't because the special vdev removal have to be strictly evacuated data in the vdev to main vdev?

I will have no choice but to alternatively use p2L2ARC for the time being.