Under low RAM zpool create might end up half mounted
#380
Comments
|
Quite curious. I usually create .vhd file to devices, so no reason it shouldn't work with OSFMount. I'll give it a try and see where the error is returned from. |
|
Tested to work just fine. It is more likely you have something open in the pool, for example if you "cd" to it. |
|
Thanks a lot for testing. I retried it, and the issues remains on my laptop. As you see, saukrs@DESKTOP-O7JE7JE MINGW64 /D/Downloads
$ sudo zpool status
pool: sample-pool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
sample-pool ONLINE 0 0 0
physicaldrive2 ONLINE 0 0 0
errors: No known data errors
saukrs@DESKTOP-O7JE7JE MINGW64 /D/Downloads
$ sudo zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
sample-pool 80M 105K 79.9M - - 3% 0% 1.00x ONLINE -
saukrs@DESKTOP-O7JE7JE MINGW64 /D/Downloads
$ sudo zfs mount
saukrs@DESKTOP-O7JE7JE MINGW64 /D/Downloads
$ sudo zfs mount | wc
0 0 0
saukrs@DESKTOP-O7JE7JE MINGW64 /D/Downloads
$ sudo zpool export sample-pool
cannot export 'sample-pool': pool is busy
Well, it's not here either. At least I don't see it: $ ll /C
total 8991898
drwxr-xr-x 1 saukrs None 0 Jan 23 2022 '$Recycle.Bin'
drwxr-xr-x 1 saukrs None 0 Apr 11 15:39 '$WinREAgent'
drwxr-xr-x 1 saukrs None 0 May 21 08:12 backups
-rw-r--r-- 1 saukrs None 186 Sep 19 2022 crisp-uninstall.bat
-rw-r--r-- 1 saukrs None 8192 May 3 13:30 DumpStack.log
-rw-r--r-- 1 saukrs None 8192 May 19 15:34 DumpStack.log.tmp
lrwxrwxrwx 1 saukrs None 31 Sep 4 2023 etc -> /c/Windows/System32/drivers/etc
-rw-r--r-- 1 saukrs None 481 Oct 10 2023 GNUmakefile
-rw-r--r-- 1 saukrs None 3184181248 May 19 15:34 hiberfil.sys
drwxr-xr-x 1 saukrs None 0 Jan 24 00:54 midipix
drwxr-xr-x 1 saukrs None 0 Nov 13 2023 MININT
drwxr-xr-x 1 saukrs None 0 May 17 01:01 msys64
drwxr-xr-x 1 saukrs None 0 Apr 29 2023 mt86plus
-rw-r--r-- 1 saukrs None 6006591488 May 21 03:47 pagefile.sys
drwxr-xr-x 1 saukrs None 0 Apr 29 2023 plop
drwxr-xr-x 1 saukrs None 0 May 3 14:45 'Program Files'
drwxr-xr-x 1 saukrs None 0 Jan 12 15:00 'Program Files (x86)'
drwxr-xr-x 1 saukrs None 0 May 18 15:08 ProgramData
drwxr-xr-x 1 saukrs None 0 Sep 15 2023 Recovery
drwxr-xr-x 1 saukrs None 0 Nov 16 2023 Restored
-rw-r--r-- 1 saukrs None 16777216 May 19 15:34 swapfile.sys
drwxr-xr-x 1 saukrs None 0 Apr 29 2023 SWSetup
-rw-r--r-- 1 saukrs None 1024 Nov 16 2023 SYSTAG.BIN
drwxr-xr-x 1 saukrs None 0 May 19 10:52 'System Volume Information'
drwxr-xr-x 1 saukrs None 0 May 7 2023 system.sav
drwxr-xr-x 1 saukrs None 0 Dec 23 20:49 Users
drwxr-xr-x 1 saukrs None 0 May 3 13:38 WindowsPS. Just moved some junk from |
|
Hmm can you |
|
OK, I did, but it doesn't seem to be getting a driveletter: saukrs@DESKTOP-O7JE7JE MINGW64 /D/Downloads
$ sudo zfs set driveletter=Z sample-pool
saukrs@DESKTOP-O7JE7JE MINGW64 /D/Downloads
$ sudo zfs mount
saukrs@DESKTOP-O7JE7JE MINGW64 /D/Downloads
$ fsutil fsinfo drives
Drives: B:\ C:\ D:\Now if try unmount + mount, the latter fails (while the former does nothing, I guess): saukrs@DESKTOP-O7JE7JE MINGW64 /D/Downloads
$ sudo zfs unmount -a
saukrs@DESKTOP-O7JE7JE MINGW64 /D/Downloads
$ sudo zfs mount -av
cannot mount 'sample-pool': Unknown errorExporting fails with the same output: saukrs@DESKTOP-O7JE7JE MINGW64 /D/Downloads
$ sudo zpool export -a
cannot export 'sample-pool': pool is busyCan I enable some debug output without running a WinDBG or other complex stuff? PS. I have Ext2Fsd and Winbtrfs software installed and running in parallel. Maybe it interferes somehow? |
|
Hmm can you reboot and import again? It has lost its mind there |
|
I did today's run in a rebooted OS session. Sure, will do again (at least to free the image file:) So no built-in debugging means to help? Putting `... get all` outputs just in case...
$ sudo zpool get all
NAME PROPERTY VALUE SOURCE
sample-pool size 80M -
sample-pool capacity 0% -
sample-pool altroot - default
sample-pool health ONLINE -
sample-pool guid 7331206367092621922 -
sample-pool version - default
sample-pool bootfs - default
sample-pool delegation on default
sample-pool autoreplace off default
sample-pool cachefile - default
sample-pool failmode wait default
sample-pool listsnapshots off default
sample-pool autoexpand off default
sample-pool dedupratio 1.00x -
sample-pool free 79.9M -
sample-pool allocated 128K -
sample-pool readonly off -
sample-pool ashift 0 default
sample-pool comment - default
sample-pool expandsize - -
sample-pool freeing 0 -
sample-pool fragmentation 5% -
sample-pool leaked 0 -
sample-pool multihost off default
sample-pool checkpoint - -
sample-pool load_guid 3759889056826721622 -
sample-pool autotrim off default
sample-pool compatibility off default
sample-pool bcloneused 0 -
sample-pool bclonesaved 0 -
sample-pool bcloneratio 1.00x -
sample-pool feature@async_destroy enabled local
sample-pool feature@empty_bpobj enabled local
sample-pool feature@lz4_compress active local
sample-pool feature@multi_vdev_crash_dump enabled local
sample-pool feature@spacemap_histogram active local
sample-pool feature@enabled_txg active local
sample-pool feature@hole_birth active local
sample-pool feature@extensible_dataset active local
sample-pool feature@embedded_data active local
sample-pool feature@bookmarks enabled local
sample-pool feature@filesystem_limits enabled local
sample-pool feature@large_blocks enabled local
sample-pool feature@large_dnode enabled local
sample-pool feature@sha512 enabled local
sample-pool feature@skein enabled local
sample-pool feature@edonr enabled local
sample-pool feature@userobj_accounting active local
sample-pool feature@encryption enabled local
sample-pool feature@project_quota active local
sample-pool feature@device_removal enabled local
sample-pool feature@obsolete_counts enabled local
sample-pool feature@zpool_checkpoint enabled local
sample-pool feature@spacemap_v2 active local
sample-pool feature@allocation_classes enabled local
sample-pool feature@resilver_defer enabled local
sample-pool feature@bookmark_v2 enabled local
sample-pool feature@redaction_bookmarks enabled local
sample-pool feature@redacted_datasets enabled local
sample-pool feature@bookmark_written enabled local
sample-pool feature@log_spacemap active local
sample-pool feature@livelist enabled local
sample-pool feature@device_rebuild enabled local
sample-pool feature@zstd_compress enabled local
sample-pool feature@draid enabled local
sample-pool feature@zilsaxattr enabled local
sample-pool feature@head_errlog active local
sample-pool feature@blake3 enabled local
sample-pool feature@block_cloning enabled local
sample-pool feature@vdev_zaps_v2 active local
sample-pool feature@redaction_list_spill enabled local
sample-pool feature@raidz_expansion enabled local
$ sudo zfs get all
NAME PROPERTY VALUE SOURCE
sample-pool type filesystem -
sample-pool creation Tue May 21 08:03 2024 -
sample-pool used 128K -
sample-pool available 39.9M -
sample-pool referenced 25.5K -
sample-pool compressratio 1.00x -
sample-pool mounted no -
sample-pool quota none default
sample-pool reservation none default
sample-pool recordsize 128K default
sample-pool mountpoint /sample-pool default
sample-pool sharenfs off default
sample-pool checksum on default
sample-pool compression on default
sample-pool atime on default
sample-pool devices on default
sample-pool exec on default
sample-pool setuid on default
sample-pool readonly off default
sample-pool zoned off default
sample-pool snapdir hidden default
sample-pool aclmode discard default
sample-pool aclinherit restricted default
sample-pool createtxg 1 -
sample-pool canmount on default
sample-pool xattr on default
sample-pool copies 1 default
sample-pool version 5 -
sample-pool utf8only off -
sample-pool normalization none -
sample-pool casesensitivity sensitive -
sample-pool vscan off default
sample-pool nbmand off default
sample-pool sharesmb off default
sample-pool refquota none default
sample-pool refreservation none default
sample-pool guid 6295728974672549137 -
sample-pool primarycache all default
sample-pool secondarycache all default
sample-pool usedbysnapshots 0B -
sample-pool usedbydataset 25.5K -
sample-pool usedbychildren 102K -
sample-pool usedbyrefreservation 0B -
sample-pool logbias latency default
sample-pool objsetid 54 -
sample-pool dedup off default
sample-pool mlslabel none default
sample-pool sync standard default
sample-pool dnodesize legacy default
sample-pool refcompressratio 1.00x -
sample-pool written 25.5K -
sample-pool logicalused 47K -
sample-pool logicalreferenced 13K -
sample-pool volmode default default
sample-pool filesystem_limit none default
sample-pool snapshot_limit none default
sample-pool filesystem_count none default
sample-pool snapshot_count none default
sample-pool snapdev hidden default
sample-pool acltype nfsv4 default
sample-pool context none default
sample-pool fscontext none default
sample-pool defcontext none default
sample-pool rootcontext none default
sample-pool relatime on default
sample-pool redundant_metadata all default
sample-pool overlay on default
sample-pool encryption off default
sample-pool keylocation none default
sample-pool keyformat none default
sample-pool pbkdf2iters 0 default
sample-pool special_small_blocks 0 default
sample-pool prefetch all default
sample-pool com.apple.mimic off default
sample-pool driveletter Z local |
|
There's a bunch of debugging, but they do require a fair bit of setup. |
|
One more thought. @lundman, when it succeeds on your machine, does it have zero OS-native partitions too? saukrs@DESKTOP-O7JE7JE MSYS /D/_BACKUPS/src
$ powershell 'Get-WmiObject -Class Win32_DiskDrive | select Name,Model,SerialNumber,InterfaceType,Partitions,Size | format-table'
Name Model SerialNumber InterfaceType Partitions Size
---- ----- ------------ ------------- ---------- ----
\\.\PHYSICALDRIVE0 Samsung SSD 870 EVO 1TB S626NF0R201463X IDE 1 1000202273280
\\.\PHYSICALDRIVE1 Samsung SSD 980 500GB 0025_38D3_32A0_5462. SCSI 4 500105249280
\\.\PHYSICALDRIVE2 PassMark osfdisk SCSI Disk Device SCSI 0 98703360 |
|
Partitions are created the ZFS way, which is all upstream stuff. It will not work right until |
|
Fair, and I was worried maybe OpenZFS needs a native OS partition for the pool to work successfully on Windows. |
|
PS. Forgot to respond to your statement:
Well, I have no idea how could I |
|
Can you also show |
|
Yes, just rebooted and recreated the virtual disk: saukrs@DESKTOP-O7JE7JE MSYS /D/Downloads
$ sudo OSFMount.com -a -t file -f openzfs-sample-pool.img -o rw,physical
Creating device...
OK
Setting disk attributes...
Done.
saukrs@DESKTOP-O7JE7JE MSYS /D/Downloads
$ sudo OSFMount.com -l
[logical disks]
[physical disks]
\\.\PhysicalDrive2Because saukrs@DESKTOP-O7JE7JE MSYS /D/Downloads
$ sudo zpool status
no pools availableNow saukrs@DESKTOP-O7JE7JE MSYS /D/Downloads
$ sudo zpool import
path '\\?\scsi#disk&ven_samsung&prod_ssd_870_evo_1tb#4&f78d1d6&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive0'
read partitions ok 2
gpt 0: type c34b6fd500 off 0x4400 len 0xffbc00
gpt 1: type c34b6fd500 off 0x1000000 len 0xe8dfc00000
asking libefi to read label
EFI read OK, max partitions 128
part 0: offset 22: len 7fde: tag: 10 name: 'Microsoft reserved partition'
part 1: offset 8000: len 746fe000: tag: 11 name: 'Basic data partition'
path '\\?\scsi#disk&ven_nvme&prod_samsung_ssd_980#5&2eacfa49&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive1'
read partitions ok 5
gpt 0: type c34b6fd500 off 0x100000 len 0x6400000
gpt 1: type c34b6fd500 off 0x6500000 len 0x1000000
gpt 2: type c34b6fd500 off 0x7500000 len 0x3b7760a600
gpt 3: type c34b6fd500 off 0x3b7ec00000 len 0x1f900000
gpt 4: type c34b6fd500 off 0x3b9e500000 len 0x38d2600000
asking libefi to read label
EFI read OK, max partitions 128
part 0: offset 800: len 32000: tag: c name: 'EFI system partition'
part 1: offset 32800: len 8000: tag: 10 name: 'Microsoft reserved partition'
part 2: offset 3a800: len 1dbbb053: tag: 11 name: 'Basic data partition'
part 4: offset 1dcf2800: len 1c693000: tag: 11 name: 'Basic data partition'
path '\\?\scsi#disk&ven_passmark&prod_osfdisk#1&2afd7d61&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive2'
read partitions ok 0
asking libefi to read label
EFI read OK, max partitions 128
no pools available to importShould I repeat creation of |
|
That's not right, why wouldn't it find the pool to import? Did it even write anything to the file hmm |
It did at least some bits. I zeroed out image file and calculated the original checksum: saukrs@DESKTOP-O7JE7JE MSYS /D/Downloads
$ sudo OSFMount.com -d -m 2
$ sudo OSFMount.com -l
[logical disks]
[physical disks]
$ dd if=/dev/zero of=./openzfs-sample-pool.img bs=1M count=100
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.0432512 s, 2.4 GB/s
$ sha1sum ./openzfs-sample-pool.img | tee openzfs-sample-pool.sha1
2c2ceccb5ec5574f791d45b63c940cff20550f9a *./openzfs-sample-pool.imgAnd recreated the $ sudo OSFMount.com -a -t file -f openzfs-sample-pool.img -o rw,physical
Creating device...
OK
Setting disk attributes...
Done.
$ sudo OSFMount.com -l
[logical disks]
[physical disks]
\\.\PhysicalDrive2No changes after import scanning: $ sudo zpool import
path '\\?\scsi#disk&ven_samsung&prod_ssd_870_evo_1tb#4&f78d1d6&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive0'
read partitions ok 2
gpt 0: type 9f594fd600 off 0x4400 len 0xffbc00
gpt 1: type 9f594fd600 off 0x1000000 len 0xe8dfc00000
asking libefi to read label
EFI read OK, max partitions 128
part 0: offset 22: len 7fde: tag: 10 name: 'Microsoft reserved partition'
part 1: offset 8000: len 746fe000: tag: 11 name: 'Basic data partition'
path '\\?\scsi#disk&ven_nvme&prod_samsung_ssd_980#5&2eacfa49&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive1'
read partitions ok 5
gpt 0: type 9f594fd600 off 0x100000 len 0x6400000
gpt 1: type 9f594fd600 off 0x6500000 len 0x1000000
gpt 2: type 9f594fd600 off 0x7500000 len 0x3b7760a600
gpt 3: type 9f594fd600 off 0x3b7ec00000 len 0x1f900000
gpt 4: type 9f594fd600 off 0x3b9e500000 len 0x38d2600000
asking libefi to read label
EFI read OK, max partitions 128
part 0: offset 800: len 32000: tag: c name: 'EFI system partition'
part 1: offset 32800: len 8000: tag: 10 name: 'Microsoft reserved partition'
part 2: offset 3a800: len 1dbbb053: tag: 11 name: 'Basic data partition'
part 4: offset 1dcf2800: len 1c693000: tag: 11 name: 'Basic data partition'
path '\\?\scsi#disk&ven_passmark&prod_osfdisk#1&2afd7d61&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive2'
read partitions ok 0
asking libefi to read label
no pools available to import
$ sha1sum -c openzfs-sample-pool.sha1
./openzfs-sample-pool.img: OKSo I recreated the testing pool: $ sudo zpool create sample-pool PhysicalDrive2
Expanded path to '\\?\PhysicalDrive2'
working on dev '#1048576#94371840#\\?\PhysicalDrive2'
setting path here '/dev/physicaldrive2'
setting physpath here '#1048576#94371840#\\?\PhysicalDrive2'
$ sudo zpool status -v
pool: sample-pool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
sample-pool ONLINE 0 0 0
physicaldrive2 ONLINE 0 0 0
errors: No known data errorsNow the image file changed: $ sha1sum -c openzfs-sample-pool.sha1
./openzfs-sample-pool.img: FAILED
sha1sum: WARNING: 1 computed checksum did NOT match
$ file openzfs-sample-pool.img
openzfs-sample-pool.img: DOS/MBR boot sector; partition 1 : ID=0xee, start-CHS (0x3ff,255,63), end-CHS (0x3ff,255,63), startsector 1, 204799 sectors, extended partition table (last)
$ fdisk -l openzfs-sample-pool.img
Disk openzfs-sample-pool.img: 100 MiB, 104857600 bytes, 204800 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: D08A4243-5817-EF11-BDD4-5FEB5555ACD2
Device Start End Sectors Size Type
openzfs-sample-pool.img1 2048 186367 184320 90M Solaris /usr & Apple ZFS
openzfs-sample-pool.img9 186368 202751 16384 8M Solaris reserved 1Looks like some partition table was written into it. And surely $ sudo zfs list
NAME USED AVAIL REFER MOUNTPOINT
sample-pool 107K 39.9M 24.5K /sample-poolSome properties: $ sudo zfs get all | grep -i -e mount -e drive
sample-pool mounted no -
sample-pool mountpoint /sample-pool default
sample-pool canmount on default
sample-pool driveletter - default
Now if I try to export the pool, it fails: saukrs@DESKTOP-O7JE7JE MSYS /D/Downloads
$ sudo zpool export sample-pool
cannot export 'sample-pool': pool is busy
saukrs@DESKTOP-O7JE7JE MSYS /D/Downloads
$ sudo zpool export -af
cannot export 'sample-pool': pool is busyJust as originally reported. |
|
Yeah you can't export it, because the mounting failed, it is half mounted, so it doesnt know to unmount. |
|
If the exact same commands (including |
|
PS. I tried the same on Linux Debian 12 using root@omn:~# losetup -l
NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE DIO LOG-SEC
/dev/loop0 0 0 0 0 /root/openzfs-sample-pool.img 0 512
root@omn:~# sha1sum -c openzfs-sample-pool.sha1
./openzfs-sample-pool.img: FAILED
sha1sum: WARNING: 1 computed checksum did NOT matchBut no partition table is written into it: root@omn:~# fdisk -l /dev/loop0
Disk /dev/loop0: 100 MiB, 104857600 bytes, 204800 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
root@omn:~# gdisk -l /dev/loop0
GPT fdisk (gdisk) version 1.0.9
Partition table scan:
MBR: not present
BSD: not present
APM: not present
GPT: not present
Creating new GPT entries in memory.
Disk /dev/loop0: 204800 sectors, 100.0 MiB
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 648FF4E6-B762-47C1-97F6-D19295B4E178
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 204766
Partitions will be aligned on 2048-sector boundaries
Total free space is 204733 sectors (100.0 MiB)
Number Start (sector) End (sector) Size Code Name... even after destroying the loop device: root@omn:~# losetup -d /dev/loop0
root@omn:~# losetup -l
root@omn:~# fdisk -l openzfs-sample-pool.img
Disk openzfs-sample-pool.img: 100 MiB, 104857600 bytes, 204800 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytesOK, I rechecked the image from Windows on Linux using saukrs@omn:~$ gdisk -l /tmp/openzfs-sample-pool.img
GPT fdisk (gdisk) version 1.0.9
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /tmp/openzfs-sample-pool.img: 204800 sectors, 100.0 MiB
Sector size (logical): 512 bytes
Disk identifier (GUID): D08A4243-5817-EF11-BDD4-5FEB5555ACD2
Partition table holds up to 9 entries
Main partition table begins at sector 2 and ends at sector 4
First usable sector is 34, last usable sector is 204766
Partitions will be aligned on 2048-sector boundaries
Total free space is 4029 sectors (2.0 MiB)
Number Start (sector) End (sector) Size Code Name
1 2048 186367 90.0 MiB BF01 zfs-000069a000004ddb
9 186368 202751 8.0 MiB BF07I also compared $ diff -ub zpool-get-all-sample-pool.windows10.txt zpool-get-all-sample-pool.linux.txt
--- zpool-get-all-sample-pool.windows10.txt 2024-05-21 14:09:14.126551200 +0300
+++ zpool-get-all-sample-pool.linux.txt 2024-05-21 14:08:48.046716800 +0300
@@ -3,7 +3,7 @@
sample-pool capacity 0% -
sample-pool altroot - default
sample-pool health ONLINE -
-sample-pool guid 7996677649176440267 -
+sample-pool guid 8533994822764781258 -
sample-pool version - default
sample-pool bootfs - default
sample-pool delegation on default
@@ -14,7 +14,7 @@
sample-pool autoexpand off default
sample-pool dedupratio 1.00x -
sample-pool free 79.9M -
-sample-pool allocated 116K -
+sample-pool allocated 110K -
sample-pool readonly off -
sample-pool ashift 0 default
sample-pool comment - default
@@ -24,7 +24,7 @@
sample-pool leaked 0 -
sample-pool multihost off default
sample-pool checkpoint - -
-sample-pool load_guid 4038385019138522589 -
+sample-pool load_guid 13219985917342397409 -
sample-pool autotrim off default
sample-pool compatibility off default
sample-pool bcloneused 0 -
@@ -69,5 +69,3 @@
sample-pool feature@blake3 enabled local
sample-pool feature@block_cloning enabled local
sample-pool feature@vdev_zaps_v2 active local
-sample-pool feature@redaction_list_spill enabled local
-sample-pool feature@raidz_expansion enabled local... and compared $ diff -ub zfs-get-all-sample-pool.windows10.txt zfs-get-all-sample-pool.linux.txt
--- zfs-get-all-sample-pool.windows10.txt 2024-05-21 14:18:52.495245400 +0300
+++ zfs-get-all-sample-pool.linux.txt 2024-05-21 14:15:13.013732300 +0300
@@ -1,11 +1,11 @@
NAME PROPERTY VALUE SOURCE
sample-pool type filesystem -
-sample-pool creation Tue May 21 12:55 2024 -
-sample-pool used 116K -
+sample-pool creation Tue May 21 14:06 2024 -
+sample-pool used 110K -
sample-pool available 39.9M -
-sample-pool referenced 24.5K -
+sample-pool referenced 24K -
sample-pool compressratio 1.00x -
-sample-pool mounted no -
+sample-pool mounted yes -
sample-pool quota none default
sample-pool reservation none default
sample-pool recordsize 128K default
@@ -35,12 +35,12 @@
sample-pool sharesmb off default
sample-pool refquota none default
sample-pool refreservation none default
-sample-pool guid 8106054128323860416 -
+sample-pool guid 10601684526528754965 -
sample-pool primarycache all default
sample-pool secondarycache all default
sample-pool usedbysnapshots 0B -
-sample-pool usedbydataset 24.5K -
-sample-pool usedbychildren 91.5K -
+sample-pool usedbydataset 24K -
+sample-pool usedbychildren 85.5K -
sample-pool usedbyrefreservation 0B -
sample-pool logbias latency default
sample-pool objsetid 54 -
@@ -49,16 +49,16 @@
sample-pool sync standard default
sample-pool dnodesize legacy default
sample-pool refcompressratio 1.00x -
-sample-pool written 24.5K -
-sample-pool logicalused 43K -
-sample-pool logicalreferenced 12.5K -
+sample-pool written 24K -
+sample-pool logicalused 40.5K -
+sample-pool logicalreferenced 12K -
sample-pool volmode default default
sample-pool filesystem_limit none default
sample-pool snapshot_limit none default
sample-pool filesystem_count none default
sample-pool snapshot_count none default
sample-pool snapdev hidden default
-sample-pool acltype nfsv4 default
+sample-pool acltype off default
sample-pool context none default
sample-pool fscontext none default
sample-pool defcontext none default
@@ -71,6 +71,3 @@
sample-pool keyformat none default
sample-pool pbkdf2iters 0 default
sample-pool special_small_blocks 0 default
-sample-pool prefetch all default
-sample-pool com.apple.mimic off default
-sample-pool driveletter - defaultNothing special. I wonder what makes OpenZFS to behave different between Linux and Windows with regard to GPT creation. Then maybe that difference is not related to the original issue. |
|
I'd be more interested to see if it makes a difference if you ran the commands from cmd/powershell started as Administrator. I can try MSYS tomorrow as well |
|
Oh, I forgot that MSYS2 + gsudo might mangle some things along (like paths). Thanks for reminding. Yes, it works indeed: Microsoft Windows [Version 10.0.19044.3086]
(c) Microsoft Corporation. All rights reserved.
C:\Windows\system32>set PROMPT=$P$G
C:\Windows\system32> PATH=%PATH%;D:\Program Files\OSFMount;C:\Program Files\OpenZFS On Windows
C:\Windows\system32> D:
D:\> cd D:\Downloads
D:\Downloads> OSFMount.com -l
[logical disks]
[physical disks]
D:\Downloads> dir openzfs-sample-pool-2.img
Volume in drive D is New Volume
Volume Serial Number is F8C2-CC01
Directory of D:\Downloads
21/05/2024 14:59 104,857,600 openzfs-sample-pool-2.img
1 File(s) 104,857,600 bytes
0 Dir(s) 323,813,376 bytes free
D:\Downloads> OSFMount.com -a -t file -f openzfs-sample-pool-2.img -o rw,physical
Creating device...
OK
Setting disk attributes...
Done.
D:\Downloads> OSFMount.com -l
[logical disks]
[physical disks]
\\.\PhysicalDrive2
D:\Downloads> zpool import
path '\\?\scsi#disk&ven_samsung&prod_ssd_870_evo_1tb#4&f78d1d6&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive0'
read partitions ok 2
gpt 0: type a90d6fd4b0 off 0x4400 len 0xffbc00
gpt 1: type a90d6fd4b0 off 0x1000000 len 0xe8dfc00000
asking libefi to read label
EFI read OK, max partitions 128
part 0: offset 22: len 7fde: tag: 10 name: 'Microsoft reserved partition'
part 1: offset 8000: len 746fe000: tag: 11 name: 'Basic data partition'
path '\\?\scsi#disk&ven_nvme&prod_samsung_ssd_980#5&2eacfa49&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive1'
read partitions ok 5
gpt 0: type a90d6fd4b0 off 0x100000 len 0x6400000
gpt 1: type a90d6fd4b0 off 0x6500000 len 0x1000000
gpt 2: type a90d6fd4b0 off 0x7500000 len 0x3b7760a600
gpt 3: type a90d6fd4b0 off 0x3b7ec00000 len 0x1f900000
gpt 4: type a90d6fd4b0 off 0x3b9e500000 len 0x38d2600000
asking libefi to read label
EFI read OK, max partitions 128
part 0: offset 800: len 32000: tag: c name: 'EFI system partition'
part 1: offset 32800: len 8000: tag: 10 name: 'Microsoft reserved partition'
part 2: offset 3a800: len 1dbbb053: tag: 11 name: 'Basic data partition'
part 4: offset 1dcf2800: len 1c693000: tag: 11 name: 'Basic data partition'
path '\\?\scsi#disk&ven_passmark&prod_osfdisk#1&2afd7d61&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive2'
read partitions ok 0
asking libefi to read label
no pools available to import
D:\Downloads> zpool create sample-pool-2 PhysicalDrive2
Expanded path to '\\?\PhysicalDrive2'
working on dev '#1048576#94371840#\\?\PhysicalDrive2'
setting path here '/dev/physicaldrive2'
setting physpath here '#1048576#94371840#\\?\PhysicalDrive2'
D:\Downloads> zpool status -v
pool: sample-pool-2
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
sample-pool-2 ONLINE 0 0 0
physicaldrive2 ONLINE 0 0 0
errors: No known data errors
D:\Downloads> zfs list
NAME USED AVAIL REFER MOUNTPOINT
sample-pool-2 116K 39.9M 27.5K /sample-pool-2
D:\Downloads> zfs mount
sample-pool-2 E:/
D:\Downloads> zpool export sample-pool-2
zunmount(sample-pool-2,E:/) running
zunmount(sample-pool-2,E:/) returns 0
D:\Downloads> zpool status
no pools available
D:\Downloads> What a waste of time! Now I need to find the offending difference (who changes what in MSYS2 scenario). |
|
It is possible it wont let you write near the start of a device? To protect partitions? |
|
I am postponing the research. This issue doesn't belong to @openzfsonwindows. Let's close it (at least for a while). |
|
OK, I might take my words back. Out of ~10 tries in elevated CMD one try failed with Looks like I am hitting some race condition. I tried to make output formats similar in both environments. When pool creation succeeded in both of them, I compared the outputs. The last two differences in the output (not in the prompts) seem strange to me: -/D/Downloads> zpool create sample-pool PhysicalDrive2
+D:\Downloads> zpool create sample-pool PhysicalDrive2
Expanded path to '\\?\PhysicalDrive2'
working on dev '#1048576#94371840#\\?\PhysicalDrive2'
setting path here '/dev/physicaldrive2'
setting physpath here '#1048576#94371840#\\?\PhysicalDrive2'
-/D/Downloads> zpool status
+D:\Downloads> zpool status
pool: sample-pool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
sample-pool ONLINE 0 0 0
physicaldrive2 ONLINE 0 0 0
errors: No known data errors
-/D/Downloads> zfs list
+D:\Downloads> zfs list
NAME USED AVAIL REFER MOUNTPOINT
-sample-pool 117K 39.9M 28.5K /sample-pool
+sample-pool 1022K 39.0M 27.5K /sample-pool
-/D/Downloads> zfs mount
+D:\Downloads> zfs mount
sample-pool E:/
-/D/Downloads> zpool list
+D:\Downloads> zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
-sample-pool 80M 117K 79.9M - - 3% 0% 1.00x ONLINE -
+sample-pool 80M 1022K 79.0M - - 3% 1% 1.00x ONLINE -Why would filesystem use 8.7x more space (1022K) when being created in the elevated CMD (the green color) in comparison to the elevated MSYS2 (117K)? @lundman, any ideas? |
|
If it does not get mounted, ie the E: line, then it will not get Explorer/Windows doing it's think with a newly mounted filesystem. It gets pretty busy when mounted. All tests that do not show E: from |
Fair enough. But as you see, it gets mounted. In my both runs (when mounting from CMD and MSYS2) Still the usage of a newly created filesystem went from 1022K (green color, run in CMD) to 117K (red color, run in MSYS2). I thought this also might have to do with the possible race condition that I just noticed. |
|
Ah I assume just one line with E: indicated the first failed, but you presented in coloured diff format. Then yeah, does seem a bit like a race bug |
|
Is it iffy from |
|
If you are asking about the last two runs, then no – Full unified diff of outputs from MSYS2 and CMD$ diff -ub10 ZFS-on-{MSYS2,CMD}-v4.txt
--- ZFS-on-MSYS2-v4.txt 2024-05-22 00:07:46.000866100 +0300
+++ ZFS-on-CMD-v4.txt 2024-05-22 00:07:15.993522200 +0300
@@ -1,118 +1,126 @@
-saukrs@DESKTOP-O7JE7JE MSYS ~
-# PS1='\n\[\e[33m\]\w\[\e[0m\]\[\e[1m\]>\[\e[0m\] '
+C:\Windows\system32>PROMPT=$P$G
-~> cd /D/Downloads
+C:\Windows\system32> cd D:\Downloads && D:
-/D/Downloads> PATH=$PATH:/C/Program\ Files/OpenZFS\ On\ Windows:/D/Program\ Files/OSFMount
+D:\Downloads> PATH=%PATH%;C:\Program Files\OpenZFS On Windows;D:\Program Files\OSFMount
-/D/Downloads> OSFMount.com -l
+D:\Downloads> PATH=%PATH%;C:\msys64\usr\bin
+
+D:\Downloads> OSFMount.com -l
[logical disks]
[physical disks]
-/D/Downloads> dd if=/dev/zero of=./openzfs-sample-pool.img bs=1M count=100
+D:\Downloads> dd if=/dev/zero of=./openzfs-sample-pool.img bs=1M count=100
100+0 records in
100+0 records out
-104857600 bytes (105 MB, 100 MiB) copied, 0.0391247 s, 2.7 GB/s
+104857600 bytes (105 MB, 100 MiB) copied, 0.0414715 s, 2.5 GB/s
-/D/Downloads> dir openzfs-sample-pool.img
+D:\Downloads> dir /b openzfs-sample-pool.img
openzfs-sample-pool.img
-/D/Downloads> ll openzfs-sample-pool.img
--rw-r--r-- 1 saukrs None 104857600 May 21 23:59 openzfs-sample-pool.img
+D:\Downloads> dir openzfs-sample-pool.img
+ Volume in drive D is New Volume
+ Volume Serial Number is F8C2-CC01
+
+ Directory of D:\Downloads
+
+21/05/2024 23:52 104,857,600 openzfs-sample-pool.img
+ 1 File(s) 104,857,600 bytes
+ 0 Dir(s) 427,872,256 bytes free
-/D/Downloads> OSFMount.com -a -t file -f openzfs-sample-pool.img -o rw,physical
+D:\Downloads> OSFMount.com -a -t file -f openzfs-sample-pool.img -o rw,physical
Creating device...
OK
Setting disk attributes...
Done.
-/D/Downloads> OSFMount.com -l
+D:\Downloads> OSFMount.com -l
[logical disks]
[physical disks]
\\.\PhysicalDrive2
-/D/Downloads> zpool import
+D:\Downloads> zpool import
path '\\?\scsi#disk&ven_samsung&prod_ssd_870_evo_1tb#4&f78d1d6&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive0'
read partitions ok 2
- gpt 0: type 76256fd7a0 off 0x4400 len 0xffbc00
- gpt 1: type 76256fd7a0 off 0x1000000 len 0xe8dfc00000
+ gpt 0: type f9775adb80 off 0x4400 len 0xffbc00
+ gpt 1: type f9775adb80 off 0x1000000 len 0xe8dfc00000
asking libefi to read label
EFI read OK, max partitions 128
part 0: offset 22: len 7fde: tag: 10 name: 'Microsoft reserved partition'
part 1: offset 8000: len 746fe000: tag: 11 name: 'Basic data partition'
path '\\?\scsi#disk&ven_nvme&prod_samsung_ssd_980#5&2eacfa49&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive1'
read partitions ok 5
- gpt 0: type 76256fd7a0 off 0x100000 len 0x6400000
- gpt 1: type 76256fd7a0 off 0x6500000 len 0x1000000
- gpt 2: type 76256fd7a0 off 0x7500000 len 0x3b7760a600
- gpt 3: type 76256fd7a0 off 0x3b7ec00000 len 0x1f900000
- gpt 4: type 76256fd7a0 off 0x3b9e500000 len 0x38d2600000
+ gpt 0: type f9775adb80 off 0x100000 len 0x6400000
+ gpt 1: type f9775adb80 off 0x6500000 len 0x1000000
+ gpt 2: type f9775adb80 off 0x7500000 len 0x3b7760a600
+ gpt 3: type f9775adb80 off 0x3b7ec00000 len 0x1f900000
+ gpt 4: type f9775adb80 off 0x3b9e500000 len 0x38d2600000
asking libefi to read label
EFI read OK, max partitions 128
part 0: offset 800: len 32000: tag: c name: 'EFI system partition'
part 1: offset 32800: len 8000: tag: 10 name: 'Microsoft reserved partition'
part 2: offset 3a800: len 1dbbb053: tag: 11 name: 'Basic data partition'
part 4: offset 1dcf2800: len 1c693000: tag: 11 name: 'Basic data partition'
path '\\?\scsi#disk&ven_passmark&prod_osfdisk#1&2afd7d61&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive2'
read partitions ok 0
asking libefi to read label
no pools available to import
-/D/Downloads> zpool status
+D:\Downloads> zpool status
no pools available
-/D/Downloads> zpool create sample-pool PhysicalDrive2
+D:\Downloads> zpool create sample-pool PhysicalDrive2
Expanded path to '\\?\PhysicalDrive2'
working on dev '#1048576#94371840#\\?\PhysicalDrive2'
setting path here '/dev/physicaldrive2'
setting physpath here '#1048576#94371840#\\?\PhysicalDrive2'
-/D/Downloads> zpool status
+D:\Downloads> zpool status
pool: sample-pool
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
sample-pool ONLINE 0 0 0
physicaldrive2 ONLINE 0 0 0
errors: No known data errors
-/D/Downloads> zfs list
+D:\Downloads> zfs list
NAME USED AVAIL REFER MOUNTPOINT
-sample-pool 117K 39.9M 28.5K /sample-pool
+sample-pool 1022K 39.0M 27.5K /sample-pool
-/D/Downloads> zfs mount
+D:\Downloads> zfs mount
sample-pool E:/
-/D/Downloads> zpool list
+D:\Downloads> zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
-sample-pool 80M 117K 79.9M - - 3% 0% 1.00x ONLINE -
+sample-pool 80M 1022K 79.0M - - 3% 1% 1.00x ONLINE -
-/D/Downloads> zpool export sample-pool
+D:\Downloads> zpool export sample-pool
zunmount(sample-pool,E:/) running
zunmount(sample-pool,E:/) returns 0
-/D/Downloads> zfs version
+D:\Downloads> zfs version
zfswin-2.2.3rc4
zfs-kmod-zfswin-2.2.3rc4
-/D/Downloads> zpool status
+D:\Downloads> zpool status
no pools available
-/D/Downloads> OSFMount.com -d -m 2
+D:\Downloads> OSFMount.com -d -m 2
-/D/Downloads> OSFMount.com -l
+D:\Downloads> OSFMount.com -l
[logical disks]
[physical disks]
-/D/Downloads> _
+D:\Downloads> _Attaching the output files: |
|
I tried running After a random iterations count (24, then 4, then 56, now 26) I see saukrs@DESKTOP-O7JE7JE MSYS ~
# time (unset N; set -e; for((;;)); do ((N=N+1)); echo -e "\nIteration $N:\n"; set -x; zpool import sample-pool && zfs mount && zpool export sample-pool; [[ $? != 0 ]] && break; set +x; done)
...
Iteration 56:
+ zpool import sample-pool
path '\\?\scsi#disk&ven_samsung&prod_ssd_870_evo_1tb#4&f78d1d6&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive0'
read partitions ok 2
gpt 0: type 2ad0d2d460 off 0x4400 len 0xffbc00
gpt 1: type 2ad0d2d460 off 0x1000000 len 0xe8dfc00000
asking libefi to read label
EFI read OK, max partitions 128
part 0: offset 22: len 7fde: tag: 10 name: 'Microsoft reserved partition'
part 1: offset 8000: len 746fe000: tag: 11 name: 'Basic data partition'
path '\\?\scsi#disk&ven_nvme&prod_samsung_ssd_980#5&2eacfa49&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive1'
read partitions ok 5
gpt 0: type 2ad0d2d460 off 0x100000 len 0x6400000
gpt 1: type 2ad0d2d460 off 0x6500000 len 0x1000000
gpt 2: type 2ad0d2d460 off 0x7500000 len 0x3b7760a600
gpt 3: type 2ad0d2d460 off 0x3b7ec00000 len 0x1f900000
gpt 4: type 2ad0d2d460 off 0x3b9e500000 len 0x38d2600000
asking libefi to read label
EFI read OK, max partitions 128
part 0: offset 800: len 32000: tag: c name: 'EFI system partition'
part 1: offset 32800: len 8000: tag: 10 name: 'Microsoft reserved partition'
part 2: offset 3a800: len 1dbbb053: tag: 11 name: 'Basic data partition'
part 4: offset 1dcf2800: len 1c693000: tag: 11 name: 'Basic data partition'
path '\\?\scsi#disk&ven_passmark&prod_osfdisk#1&2afd7d61&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
and '\\?\PhysicalDrive2'
read partitions ok 2
gpt 0: type 2ad0d2d460 off 0x100000 len 0x5a00000
gpt 1: type 2ad0d2d460 off 0x5b00000 len 0x800000
asking libefi to read label
EFI read OK, max partitions 9
part 0: offset 800: len 2d000: tag: 4 name: 'zfs-0000663a0000742a'
part 8: offset 2d800: len 4000: tag: b name: ''
working on dev '#1048576#94371840#\\?\scsi#disk&ven_passmark&prod_osfdisk#1&2afd7d61&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
setting path here '/dev/physicaldrive2'
setting physpath here '#1048576#94371840#\\?\scsi#disk&ven_passmark&prod_osfdisk#1&2afd7d61&0&000000#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
+ [[ 139 != 0 ]]
+ break
real 1m23.270s
user 0m0.908s
sys 0m3.812s
If I knew Windows Batch syntax good enough, I would run the loop here too :) |
|
Oh well, maybe I am just too low on free RAM. |
|
maybe some sleeps, so it doesnt run too fast :) |
|
Looks like it was due to memory shortage. I've run So my guess is that on low RAM conditions something breaks silently. Either in kernel mode ( |
|
Yeah, so - import might be ok, but |
sskras
changed the title
pool is busyzpool import <pool> might stall mounting in the middle
sskras
changed the title
zpool import <pool> might stall mounting in the middlezpool create <pool> <device> might stall mounting in the middle
I ran over 100 cycles of
... which was pushing RAM consumption near to 97%, that's free 241,124K out of 7,773,880K total:
I would like to stress test both |
|
Ah ok wicked. Sounds a little less urgent at least. - excellent work. |
sskras
changed the title
zpool create <pool> <device> might stall mounting in the middlezpool create might end up half mounted
|
Building the project on my own would help, I guess. At least setting up the debugging tools. Reopening for the future. |
|
I guess this tool could be used instead of the tediously running Chrome containing a gazillion of tabs: Testlimit v5.24 by Mark Russinovich, Sysinternals |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
System information
Describe the problem you're observing
I created a virtual physical drive using OSFMount. If I use it to create a new ZFS pool, I fail to either export or destroy it after that.
Describe how to reproduce the problem
The steps:
OpenZFSOnWindows-debug-2.2.3rc4.exe(switch from Read-only, switch to "Physical disk emulation", choose the "HDD" type)
zpool createfor that virtual drivezpool destroyfor that poolpool is busyerrorInclude any warning/errors/backtraces from the system logs
I use preinstalled MSYS2 and gsudo for my operations:
(I also trim the first line of MSYS2 prompt except for the first occurrence for better signal-to-noise ratio)
OSFMount is v3.1 (1003):
The text was updated successfully, but these errors were encountered: