Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Major bug in file naming (hidden files + variable suffix at the end!) on a Veracrypt drive #185

Open
PrSunflower opened this issue Oct 14, 2023 · 20 comments
Open

Major bug in file naming (hidden files + variable suffix at the end!) on a Veracrypt drive #185

PrSunflower opened this issue Oct 14, 2023 · 20 comments
Labels
bug Something isn't working feature-request Feature request

Comments

@PrSunflower
Copy link

Hello dear community, I am experiencing a bug in Curtail that is considerably slowing me down.

Before: pictures to compress

I have multiple JPEG pictures that I would like to compress, they are stored in a folder on an encrypted Veracrypt drive:

prs@PC:/media/veracrypt1$ find . -type f
./image03.JPG
./image04.JPG
./image02.JPG
./image01.JPG

During: Curtail settings

I open Curtail, I either drag-and-drop from Nautilus ; or I use the "Browse files" button in Curtail.

Current settings in "Preferences" :

image

image

Curtail does the job quickly, without any error message:

image



After: the file names are nonsense 🤦

prs@PC:/media/veracrypt1$ find . -type f
./.xdp-image01-min.JPG-VkROpH
./.xdp-image02-min.JPG-tNnCyc
./.xdp-image03-min.JPG-JeQMde
./.xdp-image04-min.JPG-jwmnVA
./image01.JPG
./image02.JPG
./image03.JPG
./image04.JPG

This is bad because:

  • These files are considered as "hidden files" in Nautilus
  • There is a random string after the extension which makes it difficult to batch rename them! (I am just a user, not a developer!)

This only happens on my Veracrypt drive

Everything seems to work well when the pictures are stored in my Linux Home Dir :

prs@PC:~/Pictures$ find . -type f
./image03.JPG
./image04.JPG
./image02.JPG
./image04-min.JPG
./image01-min.JPG
./image02-min.JPG
./image01.JPG
./image03-min.JPG

Considerations

Can you copy the pictures you want to compress in your Linux Home Dir instead ?

I cannot do that. I would like to use Curtail to compress hundreds of GiB of JPEG pictures stored on my Veracrypt drive, the drive of my Linux home Dir doesn't have that much space.


OS and software info

  • Debian trixie "testing"
  • Curtail 1.7.0 (from Flathub)
  • Veracrypt 1.25.9

Thank you for reading, any help would be greatly appreciated 🙂
@Huluti Huluti added bug Something isn't working feature-request Feature request labels Nov 3, 2023
@PrSunflower
Copy link
Author

Hello, just to say that I offer my help 🙂

If you need to test things, I can do it on my PC and share the results with you. (Also, I speak French, if it's easier for you 😉 )

@PrSunflower
Copy link
Author

PrSunflower commented May 28, 2024 •
edited
Loading

Hello, Curtail is a great tool and I would love to provide some help to solve this issue.

The bug is still forcing me to rename the files manually on my Veracrypt partition on Curtail 1.9.1:

prs@PC:/media/veracrypt1$ find . -type f
./IMG_20240528_164648_2.jpg            <<< Original files
./IMG_20240528_164655_2.jpg
./IMG_20240528_164709_3.jpg
./IMG_20240528_164711_1.jpg
./IMG_20240528_164731_2.jpg
./IMG_20240528_164755_4.jpg
./.xdp-IMG_20240528_164648_2-min.jpg-zzKpjw    <<< Compressed by Curtail
./.xdp-IMG_20240528_164655_2-min.jpg-SZNzBz
./.xdp-IMG_20240528_164709_3-min.jpg-2TtwHL
./.xdp-IMG_20240528_164711_1-min.jpg-1lXrdH
./.xdp-IMG_20240528_164731_2-min.jpg-ckjCHG
./.xdp-IMG_20240528_164755_4-min.jpg-Nlos9k

@ARAKHN1D
Copy link
Contributor

Does this happen with all image types, or only JPGs?

@Huluti
Copy link
Owner

Huluti commented Jun 1, 2024

Sorry @PrSunflower. You did a really great job in explaining the problem.
I haven't looked yet due to a serious lack of time on my part and it's not a problem that affects many people. Even though I understand that it must be annoying for you.
Thanks @ARAKHN1D for taking an interest! 🙏

@PrSunflower
Copy link
Author

Hi @Huluti , thank you for your message, I totally understand 🙂

Hi @ARAKHN1D , thanks for your help. Yes, it seems to happen to all image types :

Linux Home Dir - not affected by the bug :

<<< Original files : >>>

prs@PC:~/Curtail test HOMEDIR$ find . -type f
./Webp/ZULU Album 1 - Zulu.webp
./Webp/ZULU Album 3 - United Colors.webp
./JPGs/DSC00065.JPG
./JPGs/DSC00335.JPG
./PNGs/Red Moon Desert 2021 - By atoms83.png
./PNGs/50_years_logo_Mauritius.png

<<< After compressing with Curtail : >>>

prs@PC:~/Curtail test HOMEDIR$ find . -type f
./Webp/ZULU Album 1 - Zulu.webp
./Webp/ZULU Album 3 - United Colors.webp
./Webp/ZULU Album 1 - Zulu-min.webp
./Webp/ZULU Album 3 - United Colors-min.webp
./JPGs/DSC00335-min.JPG
./JPGs/DSC00065.JPG
./JPGs/DSC00335.JPG
./JPGs/DSC00065-min.JPG
./PNGs/Red Moon Desert 2021 - By atoms83.png
./PNGs/50_years_logo_Mauritius-min.png
./PNGs/50_years_logo_Mauritius.png
./PNGs/Red Moon Desert 2021 - By atoms83-min.png

Veracrypt encrypted partition :

prs@PC:/Curtail test VERACRYPT$ find . -type f
./JPGs/DSC00065.JPG
./JPGs/DSC00335.JPG
./PNGs/50_years_logo_Mauritius.png
./PNGs/Red Moon Desert 2021 - By atoms83.png
./Webp/ZULU Album 1 - Zulu.webp
./Webp/ZULU Album 3 - United Colors.webp

prs@PC:/Curtail test VERACRYPT$ find . -type f
./JPGs/.xdp-DSC00065-min.JPG-aNQAWn
./JPGs/.xdp-DSC00335-min.JPG-Wl1KaA
./JPGs/DSC00065.JPG
./JPGs/DSC00335.JPG
./PNGs/.xdp-50_years_logo_Mauritius-min.png.tmp-9RiVUc
./PNGs/.xdp-Red Moon Desert 2021 - By atoms83-min.png.tmp-ySGnhb
./PNGs/50_years_logo_Mauritius.png
./PNGs/Red Moon Desert 2021 - By atoms83.png
./Webp/.xdp-ZULU Album 1 - Zulu-min.webp-PFty2i
./Webp/.xdp-ZULU Album 3 - United Colors-min.webp-5zVGki
./Webp/ZULU Album 1 - Zulu.webp
./Webp/ZULU Album 3 - United Colors.webp

⚠️ I just found a workaround! ⚠️

The bug happens when I drag-and-drop my pictures from GNOME Files (Nautilus) into the Curtail window, but I just noticed that if I use the "Bulk Compress Dicrectory (Recursive)" feature inside Curtail, every file is named correctly!

prs@PC:PC/Curtail test VERACRYPT Bulk$ find . -type f
./JPGs/DSC00065.JPG
./JPGs/DSC00335.JPG
./PNGs/50_years_logo_Mauritius.png
./PNGs/Red Moon Desert 2021 - By atoms83.png
./Webp/ZULU Album 1 - Zulu.webp
./Webp/ZULU Album 3 - United Colors.webp

prs@PC:PC/Curtail test VERACRYPT Bulk$ find . -type f
./JPGs/DSC00065-min.JPG
./JPGs/DSC00065.JPG
./JPGs/DSC00335-min.JPG
./JPGs/DSC00335.JPG
./PNGs/50_years_logo_Mauritius-min.png
./PNGs/50_years_logo_Mauritius.png
./PNGs/Red Moon Desert 2021 - By atoms83-min.png
./PNGs/Red Moon Desert 2021 - By atoms83.png
./Webp/ZULU Album 1 - Zulu-min.webp
./Webp/ZULU Album 1 - Zulu.webp
./Webp/ZULU Album 3 - United Colors-min.webp
./Webp/ZULU Album 3 - United Colors.webp

So for now, I can use Bulk Compress to avoid having corrupt filenames 🙂

@ARAKHN1D
Copy link
Contributor

Hm. Seems like the issue is either with dragging and dropping or non-recursive compression. Do filenames still get corrupted for you if you turn off recursive compression in settings and drag and drop a folder?

@PrSunflower
Copy link
Author

Hi @ARAKHN1D , errrr since the folder contains folders and not pictures, Curtail's GUI just displays a blank window.

What do you want me to do? Here is the structure of my test folder:

prs@PC:/media/veracrypt1/DOWNLOADS$ tree "Curtail test VERACRYPT"
Curtail test VERACRYPT
├── JPGs
│   ├── DSC00065.JPG
│   └── DSC00335.JPG
├── PNGs
│   ├── 50_years_logo_Mauritius.png
│   └── Red Moon Desert 2021 - By atoms83.png
└── Webp
    ├── ZULU Album 1 - Zulu.webp
    └── ZULU Album 3 - United Colors.webp

4 directories, 6 files

@ARAKHN1D
Copy link
Contributor

ARAKHN1D commented Jun 20, 2024 •
edited
Loading

Curtail showing an empty window is a bug being tracked in #231. That happened because recursive compression was off, so only the inputted folders contents was searched, and not the nested folders. I would appreciate it if you could take the files out of the nested folders so it looks like this:

Curtail test VERACRYPT:
    FILE
    FILE
    FILE
    (etc...)

Then, I'd like you to turn recursive compression off, and drag and drop that folder into Curtail. This should hopefully allow us to determine the root cause of the issue.

@Huluti
Copy link
Owner

Huluti commented Oct 13, 2024 •
edited
Loading

@PrSunflower Any chance for you to check latest question from @ARAKHN1D? Thanks!

@PrSunflower
Copy link
Author

Hello, yes of course.

New structure of the test folder (inside a Veracrypt-encrypted drive)

prs@PC:/DOWNLOADS$ tree "Curtail test VERACRYPT direct"
Curtail test VERACRYPT direct
├── 50_years_logo_Mauritius.png
├── DSC00065.JPG
├── DSC00335.JPG
├── Red Moon Desert 2021 - By atoms83.png
├── ZULU Album 1 - Zulu.webp
└── ZULU Album 3 - United Colors.webp

Before

prs@PC:/DOWNLOADS$ find "Curtail test VERACRYPT direct" -type f
Curtail test VERACRYPT direct/50_years_logo_Mauritius.png
Curtail test VERACRYPT direct/DSC00065.JPG
Curtail test VERACRYPT direct/DSC00335.JPG
Curtail test VERACRYPT direct/Red Moon Desert 2021 - By atoms83.png
Curtail test VERACRYPT direct/ZULU Album 1 - Zulu.webp
Curtail test VERACRYPT direct/ZULU Album 3 - United Colors.webp

Recursive compression turned Off, folder has been drag-and-dropped :

After

prs@PC:/DOWNLOADS$ find "Curtail test VERACRYPT direct" -type f
Curtail test VERACRYPT direct/50_years_logo_Mauritius-min.png
Curtail test VERACRYPT direct/50_years_logo_Mauritius.png
Curtail test VERACRYPT direct/DSC00065-min.JPG
Curtail test VERACRYPT direct/DSC00065.JPG
Curtail test VERACRYPT direct/DSC00335-min.JPG
Curtail test VERACRYPT direct/DSC00335.JPG
Curtail test VERACRYPT direct/Red Moon Desert 2021 - By atoms83-min.png
Curtail test VERACRYPT direct/Red Moon Desert 2021 - By atoms83.png
Curtail test VERACRYPT direct/ZULU Album 1 - Zulu-min.webp
Curtail test VERACRYPT direct/ZULU Album 1 - Zulu.webp
Curtail test VERACRYPT direct/ZULU Album 3 - United Colors-min.webp
Curtail test VERACRYPT direct/ZULU Album 3 - United Colors.webp

image


Conclusion

Everything went fine here 🙂 ✔️

@ARAKHN1D
Copy link
Contributor

Hm, I thought the issue would be from either drag-and-dropping or non-recursive compression, but everything seems to be working. I could probably ask you to keep testing, but I'm not sure if that would be worth it since we found multiple ways where the app works fine. What do you think?

@PrSunflower
Copy link
Author

Hi, yeah I think the problem is when dragging-and-dropping a folder /vs/ dragging-and-dropping multiple image files, isn't it? 🙂

Drag-and-drop multiple JPG files, Recursive Compression turned ON :

prs@PC:/media/veracrypt1/DOWNLOADS/Curtail test Recursive$ find -type f
./.xdp-DSC00065-min.JPG-oEfkuR
./.xdp-DSC00335-min.JPG-LXlKNa
./DSC00065.JPG
./DSC00335.JPG

Drag-and-drop multiple JPG files, Recursive Compression turned OFF :

prs@PC:/media/veracrypt1/DOWNLOADS/Curtail test Non-recursive$ find -type f
./.xdp-DSC00065-min.JPG-fxnhGV
./.xdp-DSC00335-min.JPG-gxl38N
./DSC00065.JPG
./DSC00335.JPG

Bonus: Close and re-open Curtail, drag-and-drop multiple JPG files, Recursive Compression turned OFF :

Errrr.... What just happened? (tested multiple times by closing and re-opening Curtail, still the same)

image

Curtail version : 1.11.1


Drag-and-drop 1 folder containing the JPG files, Recursive Compression turned ON :

prs@PC:/media/veracrypt1/DOWNLOADS/Curtail test Recursive$ find -type f
./DSC00065-min.JPG
./DSC00065.JPG
./DSC00335-min.JPG
./DSC00335.JPG

Drag-and-drop 1 folder containing the JPG files, Recursive Compression turned OFF :

prs@PC:/media/veracrypt1/DOWNLOADS/Curtail test Non-recursive$ find -type f
./DSC00065-min.JPG
./DSC00065.JPG
./DSC00335-min.JPG
./DSC00335.JPG

@ARAKHN1D
Copy link
Contributor

That's good to know, thank you. We're probably doing something differently to the files when dragging and dropping multiple vs a folder, and that's causing this issue. I'll try to create a fix when I get the time.

As for the other bug you discovered, I'm not sure and I haven't been able to reproduce it. Hopefully the fix fixes both of these, but if it doesn't you should probably open another issue for it.

@ARAKHN1D
Copy link
Contributor

ARAKHN1D commented Nov 1, 2024

Did some digging, and here's what I found:

When attempting to compress files (not folders) from the drive, the path is run/user/1000/doc/[random letters and numbers]/file instead of the actual file location. This is because the file chooser portal is being used, because Curtail is a sandboxed application in a Flatpak. I think that the issue is caused by the filename being this rather than the actual filename. The issue doesn't happen with folders because the filename is run/user/1000/doc/[random letters and numbers]/folder/file`.

As for a potential solution, I'm unfortunately not sure yet. I haven't worked with portals really yet, so I'm not sure what to do. There's this function here that could maybe be used, but I don't know if we have to implement or if GTK does (there's an issue about it here). I think a potential workaround that could be done for now is to run this command:

flatpak override --user com.github.huluti.Curtail --filesystem="path/to/veracrypt/drive"

This should allow Curtail to open files directly from the specified location.

I also think #248 is related to this, since run/user/1000/doc is also popping up there. @Huluti, I'm tagging you here because I think it'll be better if you're also aware of this.

@Huluti
Copy link
Owner

Huluti commented Nov 10, 2024

Not haven't look the problem yet but in the issue you posted it seems that there is a fix used by loupe : https://gitlab.gnome.org/GNOME/gtk/-/issues/7102#note_2271015

@ARAKHN1D
Copy link
Contributor

I tried seeing if the files have that attribute when we select them, and they don't, even when the portal is being used. I'm not sure how to make the file have that attribute.

@Huluti
Copy link
Owner

Huluti commented Nov 29, 2024 •
edited
Loading

you can use that xattr, but it only works on xdg-desktop-portal new enough to have v4 of the file picker. Note that v4 is yet to land in any stable release

I know why after having asked on Matrix :)

@Huluti
Copy link
Owner

Huluti commented Nov 29, 2024

I reproduced it!
It will be like this for all files outside "home" folder.
Not happening in folder "home" thanks to --filesystem=home
Our best option in future will be this xattr::document-portal.host-path arg maybe or asking the user where to save resulting files with the FileChooser.

@Huluti Huluti mentioned this issue Nov 29, 2024
Closed
@ARAKHN1D
Copy link
Contributor

Alright. Do we wait for xattr:document-portal.host-path to land in a stable release, then?

@PrSunflower
Copy link
Author

I've seen your messages and I just wanted to say thank you for being so committed 🙂

Doing all this work out of passion, without revenue from the product itself, must be hard and I admire the Free Software philosophy. 🌠 ✌️

Thank you for sharing 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working feature-request Feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Huluti @PrSunflower @ARAKHN1D