Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

zpool status reporting inconsistent error information #16737

Open
stuartthebruce opened this issue Nov 9, 2024 · 0 comments
Open

zpool status reporting inconsistent error information #16737

stuartthebruce opened this issue Nov 9, 2024 · 0 comments
Labels
Type: Defect Incorrect behavior (e.g. crash, hang)

Comments

@stuartthebruce
Copy link

System information

Type Version/Name
Distribution Name Rocky Linux
Distribution Version 8.10
Kernel Version 4.18.0-553.22.1.el8_10.x86_64
Architecture x86_64
OpenZFS Version 2.2.6

Describe the problem you're observing

zpool status is reporting inconsistent error information

Describe how to reproduce the problem

After zpool status initially reported errors: 1 data errors, use '-v' for a list a zpool scrub was run. When that completed the same error was reported,

ZFS has finished a scrub:

  eid: 121843
class: scrub_finish
 host: zfs4
 time: 2024-11-08 18:56:33-0800
 pool: home3
state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
  see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
 scan: scrub repaired 0B in 4 days 11:37:41 with 0 errors on Fri Nov  8 18:56:33 2024
config:

	NAME                                 STATE     READ WRITE CKSUM
	home3                                ONLINE       0     0     0
	  raidz3-0                           ONLINE       0     0     0
	    wwn-0x5000cca2531dd934           ONLINE       0     0     0
	    wwn-0x5000cca2531e2a94           ONLINE       0     0     0
	    wwn-0x5000cca29187c34c           ONLINE       0     0     0
	    wwn-0x5000cca2b8482d4c           ONLINE       0     0     0
	    wwn-0x5000cca26fcec890           ONLINE       0     0     0
	    wwn-0x5000cca2531e5f6c           ONLINE       0     0     0
	    wwn-0x5000cca27003bb00           ONLINE       0     0     0
	    wwn-0x5000cca2531e84ec           ONLINE       0     0     0
	    wwn-0x5000cca2531e868c           ONLINE       0     0     0
	    wwn-0x5000cca2531e87d8           ONLINE       0     0     0
	    wwn-0x5000cca2531e9750           ONLINE       0     0     0
	    wwn-0x5000cca2b0d97f08           ONLINE       0     0     0
	    wwn-0x5000cca2531e9f48           ONLINE       0     0     0
	    wwn-0x5000cca2531eb96c           ONLINE       0     0     0
	    wwn-0x5000cca2531ec858           ONLINE       0     0     0
	  raidz3-1                           ONLINE       0     0     0
	    wwn-0x5000cca2530aa110           ONLINE       0     0     0
	    wwn-0x5000cca2530aa424           ONLINE       0     0     0
	    wwn-0x5000cca2530aacb4           ONLINE       0     0     0
	    wwn-0x5000cca2530e297c           ONLINE       0     0     0
	    wwn-0x5000cca2530f661c           ONLINE       0     0     0
	    wwn-0x5000cca291abaed0           ONLINE       0     0     0
	    wwn-0x5000cca253123c08           ONLINE       0     0     0
	    wwn-0x5000cca253158878           ONLINE       0     0     0
	    wwn-0x5000cca253168af4           ONLINE       0     0     0
	    wwn-0x5000cca2b0044388           ONLINE       0     0     0
	    wwn-0x5000cca25316d614           ONLINE       0     0     0
	    wwn-0x5000cca2849e16ec           ONLINE       0     0     0
	    wwn-0x5000cca253178e50           ONLINE       0     0     0
	    wwn-0x5000cca2531bc948           ONLINE       0     0     0
	    wwn-0x5000cca2531d7500           ONLINE       0     0     0
	  raidz3-2                           ONLINE       0     0     0
	    wwn-0x5000cca2530a4188           ONLINE       0     0     0
	    wwn-0x5000cca2530a461c           ONLINE       0     0     0
	    wwn-0x5000cca2530a4868           ONLINE       0     0     0
	    wwn-0x5000cca2530a4918           ONLINE       0     0     0
	    wwn-0x5000cca2530a4bc4           ONLINE       0     0     0
	    wwn-0x5000cca2530a58a4           ONLINE       0     0     0
	    dm-uuid-mpath-35000cca2530a63f4  ONLINE       0     0     0
	    wwn-0x5000cca2530a6adc           ONLINE       0     0     0
	    wwn-0x5000cca2530a6f3c           ONLINE       0     0     0
	    wwn-0x5000cca2530a7108           ONLINE       0     0     0
	    wwn-0x5000cca2530a7130           ONLINE       0     0     0
	    wwn-0x5000cca2530a7288           ONLINE       0     0     0
	    wwn-0x5000cca2530a7408           ONLINE       0     0     0
	    wwn-0x5000cca2530a7428           ONLINE       0     0     0
	    wwn-0x5000cca2530a7494           ONLINE       0     0     0
	  raidz3-3                           ONLINE       0     0     0
	    wwn-0x5000cca253032d58           ONLINE       0     0     0
	    scsi-35000cca253075af0           ONLINE       0     0     0
	    wwn-0x5000cca253084ee4           ONLINE       0     0     0
	    wwn-0x5000cca2aa29bc90           ONLINE       0     0     0
	    wwn-0x5000cca25308a028           ONLINE       0     0     0
	    wwn-0x5000cca253090cbc           ONLINE       0     0     0
	    wwn-0x5000cca253091b9c           ONLINE       0     0     0
	    wwn-0x5000cca2530925bc           ONLINE       0     0     0
	    wwn-0x5000cca253093758           ONLINE       0     0     0
	    wwn-0x5000cca25309e6d8           ONLINE       0     0     0
	    wwn-0x5000cca2530a0dd0           ONLINE       0     0     0
	    wwn-0x5000cca2530a0f64           ONLINE       0     0     0
	    wwn-0x5000cca2530a1140           ONLINE       0     0     0
	    wwn-0x5000cca2530a21c4           ONLINE       0     0     0
	    wwn-0x5000cca2530a23b4           ONLINE       0     0     0
	special	
	  mirror-4                           ONLINE       0     0     0
	    zfs-0ef5386f750f66d9             ONLINE       0     0     0
	    zfs-a2c8272a11a49ac3             ONLINE       0     0     0
	  mirror-6                           ONLINE       0     0     0
	    zfs-ef648f8d22311c93             ONLINE       0     0     0
	    zfs-9f637799f030adc6             ONLINE       0     0     0
	logs	
	  system-slog                        ONLINE       0     0     0
	cache
	  nvme4n1                            ONLINE       0     0     0
	  nvme5n1                            ONLINE       0     0     0

errors: 1 data errors, use '-v' for a list

And a subsequent zpool status -v also indicated an error, but did not show the file (nor a hex address pointer that I have sometimes seen),

[root@zfs4 ~]# zpool status -v
  pool: home3
 state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: scrub repaired 0B in 4 days 11:37:41 with 0 errors on Fri Nov  8 18:56:33 2024
config:

	NAME                                 STATE     READ WRITE CKSUM
	home3                                ONLINE       0     0     0
	  raidz3-0                           ONLINE       0     0     0
	    wwn-0x5000cca2531dd934           ONLINE       0     0     0
	    wwn-0x5000cca2531e2a94           ONLINE       0     0     0
	    wwn-0x5000cca29187c34c           ONLINE       0     0     0
	    wwn-0x5000cca2b8482d4c           ONLINE       0     0     0
	    wwn-0x5000cca26fcec890           ONLINE       0     0     0
	    wwn-0x5000cca2531e5f6c           ONLINE       0     0     0
	    wwn-0x5000cca27003bb00           ONLINE       0     0     0
	    wwn-0x5000cca2531e84ec           ONLINE       0     0     0
	    wwn-0x5000cca2531e868c           ONLINE       0     0     0
	    wwn-0x5000cca2531e87d8           ONLINE       0     0     0
	    wwn-0x5000cca2531e9750           ONLINE       0     0     0
	    wwn-0x5000cca2b0d97f08           ONLINE       0     0     0
	    wwn-0x5000cca2531e9f48           ONLINE       0     0     0
	    wwn-0x5000cca2531eb96c           ONLINE       0     0     0
	    wwn-0x5000cca2531ec858           ONLINE       0     0     0
	  raidz3-1                           ONLINE       0     0     0
	    wwn-0x5000cca2530aa110           ONLINE       0     0     0
	    wwn-0x5000cca2530aa424           ONLINE       0     0     0
	    wwn-0x5000cca2530aacb4           ONLINE       0     0     0
	    wwn-0x5000cca2530e297c           ONLINE       0     0     0
	    wwn-0x5000cca2530f661c           ONLINE       0     0     0
	    wwn-0x5000cca291abaed0           ONLINE       0     0     0
	    wwn-0x5000cca253123c08           ONLINE       0     0     0
	    wwn-0x5000cca253158878           ONLINE       0     0     0
	    wwn-0x5000cca253168af4           ONLINE       0     0     0
	    wwn-0x5000cca2b0044388           ONLINE       0     0     0
	    wwn-0x5000cca25316d614           ONLINE       0     0     0
	    wwn-0x5000cca2849e16ec           ONLINE       0     0     0
	    wwn-0x5000cca253178e50           ONLINE       0     0     0
	    wwn-0x5000cca2531bc948           ONLINE       0     0     0
	    wwn-0x5000cca2531d7500           ONLINE       0     0     0
	  raidz3-2                           ONLINE       0     0     0
	    wwn-0x5000cca2530a4188           ONLINE       0     0     0
	    wwn-0x5000cca2530a461c           ONLINE       0     0     0
	    wwn-0x5000cca2530a4868           ONLINE       0     0     0
	    wwn-0x5000cca2530a4918           ONLINE       0     0     0
	    wwn-0x5000cca2530a4bc4           ONLINE       0     0     0
	    wwn-0x5000cca2530a58a4           ONLINE       0     0     0
	    dm-uuid-mpath-35000cca2530a63f4  ONLINE       0     0     0
	    wwn-0x5000cca2530a6adc           ONLINE       0     0     0
	    wwn-0x5000cca2530a6f3c           ONLINE       0     0     0
	    wwn-0x5000cca2530a7108           ONLINE       0     0     0
	    wwn-0x5000cca2530a7130           ONLINE       0     0     0
	    wwn-0x5000cca2530a7288           ONLINE       0     0     0
	    wwn-0x5000cca2530a7408           ONLINE       0     0     0
	    wwn-0x5000cca2530a7428           ONLINE       0     0     0
	    wwn-0x5000cca2530a7494           ONLINE       0     0     0
	  raidz3-3                           ONLINE       0     0     0
	    wwn-0x5000cca253032d58           ONLINE       0     0     0
	    scsi-35000cca253075af0           ONLINE       0     0     0
	    wwn-0x5000cca253084ee4           ONLINE       0     0     0
	    wwn-0x5000cca2aa29bc90           ONLINE       0     0     0
	    wwn-0x5000cca25308a028           ONLINE       0     0     0
	    wwn-0x5000cca253090cbc           ONLINE       0     0     0
	    wwn-0x5000cca253091b9c           ONLINE       0     0     0
	    wwn-0x5000cca2530925bc           ONLINE       0     0     0
	    wwn-0x5000cca253093758           ONLINE       0     0     0
	    wwn-0x5000cca25309e6d8           ONLINE       0     0     0
	    wwn-0x5000cca2530a0dd0           ONLINE       0     0     0
	    wwn-0x5000cca2530a0f64           ONLINE       0     0     0
	    wwn-0x5000cca2530a1140           ONLINE       0     0     0
	    wwn-0x5000cca2530a21c4           ONLINE       0     0     0
	    wwn-0x5000cca2530a23b4           ONLINE       0     0     0
	special	
	  mirror-4                           ONLINE       0     0     0
	    zfs-0ef5386f750f66d9             ONLINE       0     0     0
	    zfs-a2c8272a11a49ac3             ONLINE       0     0     0
	  mirror-6                           ONLINE       0     0     0
	    zfs-ef648f8d22311c93             ONLINE       0     0     0
	    zfs-9f637799f030adc6             ONLINE       0     0     0
	logs	
	  system-slog                        ONLINE       0     0     0
	cache
	  nvme4n1                            ONLINE       0     0     0
	  nvme5n1                            ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

Howwever, another zpool status indicated there are no errors (which I believe is the truth),

[root@zfs4 ~]# zpool status
  pool: home3
 state: ONLINE
  scan: scrub repaired 0B in 4 days 11:37:41 with 0 errors on Fri Nov  8 18:56:33 2024
config:

	NAME                                 STATE     READ WRITE CKSUM
	home3                                ONLINE       0     0     0
	  raidz3-0                           ONLINE       0     0     0
	    wwn-0x5000cca2531dd934           ONLINE       0     0     0
	    wwn-0x5000cca2531e2a94           ONLINE       0     0     0
	    wwn-0x5000cca29187c34c           ONLINE       0     0     0
	    wwn-0x5000cca2b8482d4c           ONLINE       0     0     0
	    wwn-0x5000cca26fcec890           ONLINE       0     0     0
	    wwn-0x5000cca2531e5f6c           ONLINE       0     0     0
	    wwn-0x5000cca27003bb00           ONLINE       0     0     0
	    wwn-0x5000cca2531e84ec           ONLINE       0     0     0
	    wwn-0x5000cca2531e868c           ONLINE       0     0     0
	    wwn-0x5000cca2531e87d8           ONLINE       0     0     0
	    wwn-0x5000cca2531e9750           ONLINE       0     0     0
	    wwn-0x5000cca2b0d97f08           ONLINE       0     0     0
	    wwn-0x5000cca2531e9f48           ONLINE       0     0     0
	    wwn-0x5000cca2531eb96c           ONLINE       0     0     0
	    wwn-0x5000cca2531ec858           ONLINE       0     0     0
	  raidz3-1                           ONLINE       0     0     0
	    wwn-0x5000cca2530aa110           ONLINE       0     0     0
	    wwn-0x5000cca2530aa424           ONLINE       0     0     0
	    wwn-0x5000cca2530aacb4           ONLINE       0     0     0
	    wwn-0x5000cca2530e297c           ONLINE       0     0     0
	    wwn-0x5000cca2530f661c           ONLINE       0     0     0
	    wwn-0x5000cca291abaed0           ONLINE       0     0     0
	    wwn-0x5000cca253123c08           ONLINE       0     0     0
	    wwn-0x5000cca253158878           ONLINE       0     0     0
	    wwn-0x5000cca253168af4           ONLINE       0     0     0
	    wwn-0x5000cca2b0044388           ONLINE       0     0     0
	    wwn-0x5000cca25316d614           ONLINE       0     0     0
	    wwn-0x5000cca2849e16ec           ONLINE       0     0     0
	    wwn-0x5000cca253178e50           ONLINE       0     0     0
	    wwn-0x5000cca2531bc948           ONLINE       0     0     0
	    wwn-0x5000cca2531d7500           ONLINE       0     0     0
	  raidz3-2                           ONLINE       0     0     0
	    wwn-0x5000cca2530a4188           ONLINE       0     0     0
	    wwn-0x5000cca2530a461c           ONLINE       0     0     0
	    wwn-0x5000cca2530a4868           ONLINE       0     0     0
	    wwn-0x5000cca2530a4918           ONLINE       0     0     0
	    wwn-0x5000cca2530a4bc4           ONLINE       0     0     0
	    wwn-0x5000cca2530a58a4           ONLINE       0     0     0
	    dm-uuid-mpath-35000cca2530a63f4  ONLINE       0     0     0
	    wwn-0x5000cca2530a6adc           ONLINE       0     0     0
	    wwn-0x5000cca2530a6f3c           ONLINE       0     0     0
	    wwn-0x5000cca2530a7108           ONLINE       0     0     0
	    wwn-0x5000cca2530a7130           ONLINE       0     0     0
	    wwn-0x5000cca2530a7288           ONLINE       0     0     0
	    wwn-0x5000cca2530a7408           ONLINE       0     0     0
	    wwn-0x5000cca2530a7428           ONLINE       0     0     0
	    wwn-0x5000cca2530a7494           ONLINE       0     0     0
	  raidz3-3                           ONLINE       0     0     0
	    wwn-0x5000cca253032d58           ONLINE       0     0     0
	    scsi-35000cca253075af0           ONLINE       0     0     0
	    wwn-0x5000cca253084ee4           ONLINE       0     0     0
	    wwn-0x5000cca2aa29bc90           ONLINE       0     0     0
	    wwn-0x5000cca25308a028           ONLINE       0     0     0
	    wwn-0x5000cca253090cbc           ONLINE       0     0     0
	    wwn-0x5000cca253091b9c           ONLINE       0     0     0
	    wwn-0x5000cca2530925bc           ONLINE       0     0     0
	    wwn-0x5000cca253093758           ONLINE       0     0     0
	    wwn-0x5000cca25309e6d8           ONLINE       0     0     0
	    wwn-0x5000cca2530a0dd0           ONLINE       0     0     0
	    wwn-0x5000cca2530a0f64           ONLINE       0     0     0
	    wwn-0x5000cca2530a1140           ONLINE       0     0     0
	    wwn-0x5000cca2530a21c4           ONLINE       0     0     0
	    wwn-0x5000cca2530a23b4           ONLINE       0     0     0
	special	
	  mirror-4                           ONLINE       0     0     0
	    zfs-0ef5386f750f66d9             ONLINE       0     0     0
	    zfs-a2c8272a11a49ac3             ONLINE       0     0     0
	  mirror-6                           ONLINE       0     0     0
	    zfs-ef648f8d22311c93             ONLINE       0     0     0
	    zfs-9f637799f030adc6             ONLINE       0     0     0
	logs	
	  system-slog                        ONLINE       0     0     0
	cache
	  nvme4n1                            ONLINE       0     0     0
	  nvme5n1                            ONLINE       0     0     0

errors: No known data errors

Include any warning/errors/backtraces from the system logs

During and after the zpool scrub no low-level system/device error messages where recorded.
@stuartthebruce stuartthebruce added the Type: Defect Incorrect behavior (e.g. crash, hang) label Nov 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Type: Defect Incorrect behavior (e.g. crash, hang)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@stuartthebruce