zvol: Enable zvol threading functionality on FreeBSD

[fuporovvStack]

Motivation and Context

Add zvol IO taskqs support on FreeBSD side. Make zvol threadpool creation/destruction logic and zvol module parameters shared for both Linux and FreeBSD.

Description

Make zvol I/O requests processing asynchronous on FreeBSD side in some cases. Clone zvol threading logic and required module parameters from Linux side. Make zvol threadpool creation/destruction logic shared for both Linux and FreeBSD.
The IO requests are processed asynchronously in next cases:

• volmode=geom: if IO request passed thru zvol_geom_worker thread.
• volmode=cdev: if IO request passed thru struct cdevsw .d_strategy routine, mean is AIO request.

In all other cases the IO requests are processed synchronously.

How Has This Been Tested?

The zvol_request_sync module parameter manipulation was added to tests/functional/zvol/zvol_stress/zvol_stress.ksh testcase for both Linux and FreeBSD.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[fuporovvStack]

@amotin, @markjdb

[amotin]

I am happy to see somebody working on unifying this code. But see some comments inline:

[fuporovvStack]

I found 4 ways to call zvol, and decided to use next sync/async switching logic dependent of volmode parameter:
geom:
- (1) zvol_geom_bio_start() - sync
- (2) zvol_geom_worker() - async
cdev:
- (3) zvol_cdev_read()/_write() - sync
- (4) zvol_geom_bio_strategy() - async

(1) - the way dependent from geom thread stack usage, case when direct=1.
(2) - case when direct variable value become zero. The direct value is not depend from sync or aio usage.
(3) - normal usage in cdev mode, example: fio with engine=sync.
(4) - used in case of AIO, example: fio with engine=posixaio

[amotin]

(1) - the way dependent from geom thread [stack usage](https://github.com/freebsd/freebsd-src/blob/main/sys/geom/geom_io.c#L566), case when direct=1.

Aside of stack usage, some GEOM classes may be unable to use direct I/O due to locking constraints and have to switch into GEOM up/down threads.

[tonyhutter]

@fuporovvStack can you rebase on the latest master so you pull in d947b9a?