Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NAS-131081 / 24.10 / Sync stable/electriceel branch with openzfs/master #253

Merged
merged 12 commits into from
Sep 10, 2024
Merged

NAS-131081 / 24.10 / Sync stable/electriceel branch with openzfs/master #253

merged 12 commits into from
Sep 10, 2024

Conversation

usaleem-ix
Copy link

@usaleem-ix usaleem-ix commented Sep 10, 2024 •
edited by amotin
Loading

Motivation and Context

Sync stable/electriceel with openzfs/master.

This PR merges dedup patches and re-enables mount.zfs fix to mount legacy datasets with correct mount options.

Description

How Has This Been Tested?

Created a 24.10 image with updates included in this PR. Manually installed and booted the image to confirm there are no issues.

Custom build artifacts are present here. API tests run can be found here.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

PHSpline and others added 12 commits September 3, 2024 17:52
@PHSpline
Remove unused sysctl node
bf8c61f 
PR openzfs#14953 removed vdev-level read cache but accidentally left this
sysctl node behind.

Reviewed-by: Rich Ercolani <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: Alexander Motin <[email protected]>
Signed-off-by: Seth Hoffert <[email protected]>
Closes openzfs#16493
@robn @behlendorf
zdb: rework dedup accounting for log, quota and prune
4a4f7b0 
The simplest thing first: add the FDT and log objects to the list of
objects to be considered when checking for leaks.

The rest is based on a conceptual change in all of this patch stack: a
block on disk with a 'D' bit is not necessarily in the DDT at all
(pruned), or in the DDT ZAPs (still on the log).

As such, walking the DDT up front is difficult (for all the reasons that
walking an unflushed log is difficult) and not really useful, since it's
not a reflection of what's on disk anyway.

Instead, we rework things here to be more like the BRT checks. When we
see a dedup'd block, we look it up in the DDT, consume a refcount, and
for the second-or-later instances, count them as duplicates.

Reviewed-by: Alexander Motin <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Co-authored-by: Allan Jude <[email protected]>
Co-authored-by: Don Brady <[email protected]>
Signed-off-by: Rob Norris <[email protected]>
Sponsored-by: Klara, Inc.
Sponsored-by: iXsystems, Inc.
Closes openzfs#16277
@don-brady @behlendorf
Add DDT prune command
d4d7945 
Requires the new 'flat' physical data which has the start
time for a class entry.

The amount to prune can be based on a target percentage of
the unique entries or based on the age (i.e., every entry
older than N days).

Sponsored-by: Klara, Inc.
Sponsored-by: iXsystems, Inc.
Reviewed-by: Alexander Motin <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Signed-off-by: Don Brady <[email protected]>
Closes openzfs#16277
@robn @tonyhutter
value strings: pretty printers for flags and enums
82ff9aa 
This adds zfs_valstr, a collection of pretty printers for bitfields and
enums. These are useful in debugging, logging and other display contexts
where raw values are difficult for the untrained (or even trained!) eye
to decipher.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Signed-off-by: Rob Norris <[email protected]>
@robn @tonyhutter
zpool events: expand value strings for ZIO error values
17dd66d 
Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Signed-off-by: Rob Norris <[email protected]>
@robn
spa_prop_get: require caller to supply output nvlist
b109925 
All callers to spa_prop_get() and spa_prop_get_nvlist() supplied their
own preallocated nvlist (except ztest), so we can remove the option to
have them allocate one if none is supplied.

This sidesteps a bug in spa_prop_get(), where the error var wasn't
initialised, which could lead to the provided nvlist being freed at the
end.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed-by: Jorgen Lundman <[email protected]>
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: Alexander Motin <[email protected]>
Signed-off-by: Rob Norris <[email protected]>
Closes openzfs#16505
@robn
libzstd: also build with LIBZPOOL_CPPFLAGS
5c67820 
libzstd now also allocates its own abd_t, and so has the same issue as
zstream did, so this applies the same workaround: compile it with
ZFS_DEBUG. See 92fca1c.

This looks weird, because libzstd doesn't appear to look related to the
ZFS kernel, but there is already a cross-dependency there: zstd needs
zfs_lz4_compress, and zfs needs zfs_zstd_compress (and others), so the
two can never really be separated without more work. Another job for
another time.

Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.
Reviewed by: Brian Behlendorf <[email protected]>
Reviewed-by: Mark Maybee <[email protected]>
Signed-off-by: Rob Norris <[email protected]>
Closes openzfs#16489
@robn
zio_resume: log when unsuspending the pool (openzfs#16485)
8be2f4c 
When reviewing logs after a failure, its useful to see where
unsuspend/resume was requested.


Sponsored-by: Klara, Inc.
Sponsored-by: Wasabi Technology, Inc.

Signed-off-by: Rob Norris <[email protected]>
Reviewed-by: Alexander Motin <[email protected]>
Reviewed-by: Tony Hutter <[email protected]>
@jrtc27
sys/types32.h: Remove struct timeval32 from libspl's header (openzfs#…
88433e6 
…16491)

macOS Sequoia's sys/sockio.h, as included by various bootstrap tools
whilst building FreeBSD, has started to include net/if.h, which then
includes sys/_types/_timeval32.h and provide a conflicting definition
for struct timeval32. Since this type is entirely unused within OpenZFS,
simply delete the type rather than adding in some kind of OS detection.

This fixes building FreeBSD on macOS Sequoia (Beta).

Signed-off-by: Jessica Clarke <[email protected]>
Reviewed-by: Rob Norris <[email protected]>
Reviewed-by: Alexander Motin <[email protected]>
Reviewed-by: Tony Hutter <[email protected]>
@robn
zts-report: don't crash on non-UTF-8 chars in the log (openzfs#16497)
63253db 
The report generator expects the log to be clean and tidy UTF-8. That
can be a problem if you use some of the verbose/debug test runner
options, which sends all sorts of weird output from arbitrary programs
to the log.

This just makes Python a little more relaxed about such things. It
shouldn't matter in practice, as those lines didn't match the test
result regex anyway, and are discarded immediately.


Sponsored-by: https://despairlabs.com/sponsor/

Signed-off-by: Rob Norris <[email protected]>
Reviewed-by: Alexander Motin <[email protected]>
Reviewed-by: Tony Hutter <[email protected]>
@usaleem-ix
Revert "Revert "Make mount.zfs(8) calling zfs_mount_at for legacy
927f6ea 
mounts""

This reverts commit f914964.

Signed-off-by: Umer Saleem <[email protected]>
@usaleem-ix
Merge branch 'master' into NAS-130821-2
80110e7 
Signed-off-by: Umer Saleem <[email protected]>
@usaleem-ix usaleem-ix changed the title NAS-130821 / 24.10 / Sync stable/electriceel branch with openzfs/master NAS-131081 / 24.10 / Sync stable/electriceel branch with openzfs/master Sep 10, 2024
@truenas truenas deleted a comment from bugclerk Sep 10, 2024
@amotin amotin merged commit 3b73b0e into stable/electriceel Sep 10, 2024
20 of 22 checks passed
@bugclerk
Copy link

JIRA ticket https://ixsystems.atlassian.net/browse/NAS-131081 is targeted to the following versions which have not received their corresponding PRs: 24.10-RC.1

@bugclerk
Copy link

This PR has been merged and conversations have been locked.
If you would like to discuss more about this issue please use our forums or raise a Jira ticket.

@truenas truenas locked as resolved and limited conversation to collaborators Sep 10, 2024
@amotin amotin deleted the NAS-131081-EE branch September 10, 2024 14:40
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@amotin amotin amotin approved these changes

Assignees
No one assigned
Labels
no-time-tracked
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@usaleem-ix @bugclerk @amotin @PHSpline @robn @don-brady @jrtc27