syncfs: don't erroneously return success when the pool is suspended

[robn]

[Sponsors: Klara, Inc., Wasabi Technology, Inc.]

Motivation and Context

See #17416. This PR is tackling the core problem described: syncfs() can return success even when the pool has failed. There's a lot more we can do but this should address correctness, which is more important than anything else.

I'm hesitant to say this closes #17416, since there's more we could do in there to improve the situation. On the other hand, it straight-up says "this is wrong" and this PR un-wrongs it, so idk, reviewer's call :)

Description

See individual commits, described below.

1. Add a test case to demonstrate the issue, and verify the fix. We don't generally care what syncfs() returns when the pool is up, but we want to be sure that if the pool suspends, it either blocks until it returns, or returns an error. (we do care, of course, but it's not something we can address here; I'm working towards that in ZIL: "crash" the ZIL if the pool suspends during fallback #17398 and later).

2. Simplify zfs_sync() by removing code for scenarios that can't occur, that is, called with NULL superblock and called with no ZIL. See syncfs() returns success on suspended pools #17416 for more on this.

3. Change the suspended check to return EIO if the pool suspends. This is nod back to the original reason for this check, which is to not block on suspended pools during shutdown (see discussion in syncfs() returns success on suspended pools #17416). This will likely change in ZIL: "crash" the ZIL if the pool suspends during fallback #17398 once the ZIL can report a failure directly without blocking, but until then, this seems like a reasonable middle road to not block the call.

4. Ensure safe error behaviour according to what syncfs() can do. The comment explains more, but the short version is that before 5.17 the kernel ignores errors directly returned by sync_fs(), but a workaround is available back to 5.8. If the workaround is not available either, then the call will return success to userspace, and we have no choice but to block until the txg completes. Unamazing, but ensures correctness.

How Has This Been Tested?

The new syncfs_suspend test passes on 5.4.293 (blocks, no fallbacks available) and 6.1.140 (sync_fs return works, immediate error return).

It's difficult to test the superblock writeback error case on a release kernel, since the LTS kernels in the 5.8-5.17 range that have no other options all have the fix backported, and non-LTS 5.x kernels are difficult to build these days. To check, I used 6.1.140, commented out the version check and forced the returned error to 0, leaving the superblock writeback error as the only remaining error state. This went as expected - syncfs() return an error on a suspended pool.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Quality assurance (non-breaking change which makes the code more robust against bugs)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[pcd1193182]

The solution is sort of gross, but so it the problem space. I can't think of a better approach than this one, given the constraints we have. I would say you could close the github issue, though I would consider opening a new one specifically for "We can't tell if the system is shutting down in the syncfs() path"