Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash during initial upload #7

Open
kallekn opened this issue Mar 28, 2024 · 30 comments
Open

Crash during initial upload #7

kallekn opened this issue Mar 28, 2024 · 30 comments

Comments

@kallekn
Copy link

kallekn commented Mar 28, 2024

I got the plugin working on my Mac, but after loading a bit more than 1 000 files it crashed, I just got repeated error messages. I closed Obsidian and opened it again, the plugin seems to be running but no new uploads are happening. Is there any way to restart the uploads, other than selecting "upload all", which would mean deleting the alreadu uploaded 1 000 files.

I have more than 6 000 files to upload, so this seems a bit shaky.

Also, after this the whole app started misbehaving. Impossible to display notes. When closing down the app I got following:

Uncaught Exception:
TypeError: Cannot read properties of undefined (reading 'origin')
at Function. (/Users/kalle.kniivila/Applications/Obsidian.app/Contents/Resources/obsidian.asar/main.js:27:9882)

After starting the app the problem persists, I can read one note, but when I try to switch to another one, I just get the title of the other note, with the content of the first one. Then nothing will work at all.

Turning the plugin off didn't seem to help. After removing the plugin files Obsidian seems to work normally again. But no sync :-(

The file that caused the crash might be a sound file, as that is the last file uploaded and it is zero bytes.

@stravo1
Copy link
Owner

stravo1 commented Mar 29, 2024

I have personally not tested the plug-in for vaults over 1000 files, will do it soon.

And no you don't need to re-upload all files, if you edit a file (notes only, not attachments) which was not uploaded it will ask for a confirmation and upload automatically, however if it's an attachment which was not uploaded, you need to make use of the "upload" button to upload them manually.

I will try and add a functionality to resume the upload process if the initial upload process fails, so that this type of issues can be avoided.

And for the crashing part, please enable the "Verbose Logging" and "Error Logging" in the plug-in settings and attach them here.

@stravo1
Copy link
Owner

stravo1 commented Mar 29, 2024

Also would suggest deleting the 0 byte file from Drive (delete from Drive's Bin as well) and trying to use the plug-in again.

@kallekn
Copy link
Author

kallekn commented Mar 29, 2024

I have personally not tested the plug-in for vaults over 1000 files, will do it soon.

And no you don't need to re-upload all files, if you edit a file (notes only, not attachments) which was not uploaded it will ask for a confirmation and upload automatically, however if it's an attachment which was not uploaded, you need to make use of the "upload" button to upload them manually.

I will try and add a functionality to resume the upload process if the initial upload process fails, so that this type of issues can be avoided.

And for the crashing part, please enable the "Verbose Logging" and "Error Logging" in the plug-in settings and attach them here.

Thanks for the quick answer. Looks like I'll need to reload everything. Too much work doing edits in 5000 files...

Some options for what should be synced would be great in the future. Like "sync stuff which was modified during last month".

But maybe I'll just put some of the old notes in an archive vault...

@kallekn
Copy link
Author

kallekn commented Mar 29, 2024

Ok, I got it working on my Mac when I reduced the vault to approximately 500 files.

However, on my Android phone the initial download process crashes after approximately 400 files. I just get ERROR: Something went wrong! Sync is paused. - over and over again.

Also, it is a bit irritating that the warning totally covers the top menu, so it is difficult to hit the "settings" icon in order to turn off the plugin.

@stravo1
Copy link
Owner

stravo1 commented Mar 29, 2024

Yes, the infinite error is a bug i am looking into. Will keep you updated.

@stravo1
Copy link
Owner

stravo1 commented Mar 29, 2024

Also can you please include the verbose and error logs.

@stravo1
Copy link
Owner

stravo1 commented Mar 30, 2024

Yes, the infinite error is a bug i am looking into. Will keep you updated.

fixed at https://github.com/stravo1/obsidian-gdrive-sync/releases/tag/v0.9.9-beta-7-1

@kallekn
Copy link
Author

kallekn commented Mar 30, 2024

Thanks. Seems to be working now, I succeeded in syncing the files to my phone.

However, there seems to be a problem with metadata: the "created" date is not the original date, so I cannot sort the notes according to how old or new they are.

Also, all attachment files seem to have been renamed, so all the links in my notes are broken :-(

@kallekn
Copy link
Author

kallekn commented Mar 30, 2024

Also can you please include the verbose and error logs.

I'll do that next time I get errors.

@kallekn
Copy link
Author

kallekn commented Mar 30, 2024

So, here are some error logs. Now I tried to sync the preexisting notes to my other Mac laptop. Not very successful, I'm afraid.

error-log-gdrive-plugin.md
verbose-log-gdrive-plugin.md

@stravo1
Copy link
Owner

stravo1 commented Mar 31, 2024

Thanks. Seems to be working now, I succeeded in syncing the files to my phone.

However, there seems to be a problem with metadata: the "created" date is not the original date, so I cannot sort the notes according to how old or new they are.

Also, all attachment files seem to have been renamed, so all the links in my notes are broken :-(

sorry for not mentioning it in the README file, yes the attachments are renamed when they are uploaded because unlike notes there's no "lastSync" tag that the plug-in can read from the note's content, so to differentiate between which attachments are synced and which are not it's renamed to "attachment_name-synced". i am actively trying to create a workaround for this, but it will take time.

there're a lot of bugs which too might lead to data loss, please do make a copy of the vault before installing the plug-in.

will make sure to add this warning on the README page.

@stravo1
Copy link
Owner

stravo1 commented Mar 31, 2024

So, here are some error logs. Now I tried to sync the preexisting notes to my other Mac laptop. Not very successful, I'm afraid.

error-log-gdrive-plugin.md
verbose-log-gdrive-plugin.md

took a look into the logs, couldn't decipher much, but seems like there's some conflict in the folder structure, if there's nothing sensitive in the vault which gave these errors, can you share the vault structure as well as a screenshot of the Drive folder where this vault was uploaded.

@kallekn
Copy link
Author

kallekn commented Mar 31, 2024

There might have been a conflict with another syncing app that was running. I'll test again and come back to you with the results.

@kallekn
Copy link
Author

kallekn commented Apr 2, 2024

Still not working properly. Created new note on my phone, made an audio recording in it. After sync the audio recording cannot be found.Screenshot_20240402_220620_Obsidian.jpg

Tue Apr 02 2024 20:47:16 GMT+0200 (Keski-Euroopan kesäaika)-TypeError-Cannot read properties of null (reading '0')-TypeError: Cannot read properties of null (reading '0')
at driveSyncPlugin.uploadNewAttachment (plugin:obsidian-gdrive-sync:3004:161)
at async eval (plugin:obsidian-gdrive-sync:3318:13)

@stravo1
Copy link
Owner

stravo1 commented Apr 3, 2024

The recording was probably synced and was renamed to:
"Recording....-synced.webm" because as I mentioned earlier attachments are renamed to keep track which are synced and which are not.

The error occurs while trying to show you a pop-up which tells you to rename the attachment in your notes file as well. I have identified the issue and will try and fix it as soon as possible.

Screenshot_20240403-091735.png

Screenshot_20240403-091726.png

@kallekn
Copy link
Author

kallekn commented Apr 3, 2024

Yes, there is a sound file with the new name. This is highly impractical, as all links to files in my notes are broken. I do hope you can find another way to do this.

@kallekn
Copy link
Author

kallekn commented Apr 3, 2024

This is what I get trying to sync my laptop now:

Wed Apr 03 2024 09:02:32 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:39 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:46 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:52 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:58 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:03:04 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:03:10 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:03:12 GMT+0200 (Keski-Euroopan kesäaika)-Error-FATAL ERROR: Too many errors within a minute.-Error: FATAL ERROR: Too many errors within a minute.
at eval (plugin:obsidian-gdrive-sync:2782:42)

@stravo1
Copy link
Owner

stravo1 commented Apr 3, 2024

the attachments are renamed when they are uploaded because unlike notes there's no "lastSync" tag that the plug-in can read from the note's content, so to differentiate between which attachments are synced and which are not it's renamed to "attachment_name-synced". i am actively trying to create a workaround for this, but it will take time.

I understand, for new vaults it's an annoyance but for big existing vaults it's a deal breaker. for the This hack works for small vaults that I personally use without too many attachments so I did not see the need to give too much attention to this issue. Now that others are facing this issue, I will try and figure out a proper solution, however it will take time.

@stravo1
Copy link
Owner

stravo1 commented Apr 3, 2024

This is what I get trying to sync my laptop now:

Wed Apr 03 2024 09:02:32 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:39 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:46 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:52 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:02:58 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:03:04 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:03:10 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists. at t. (app://obsidian.md/app.js:1:736472) at app://obsidian.md/app.js:1:237056 at Object.next (app://obsidian.md/app.js:1:237161) at a (app://obsidian.md/app.js:1:235879)

Wed Apr 03 2024 09:03:12 GMT+0200 (Keski-Euroopan kesäaika)-Error-FATAL ERROR: Too many errors within a minute.-Error: FATAL ERROR: Too many errors within a minute. at eval (plugin:obsidian-gdrive-sync:2782:42)

its the same issue as in the error logs, there's some conflict between folders, can't diagnose only from the error logs unfortunately. you can contact me on my email [email protected] for further assistance on this issue, i might need to look at the vault structure.

@stravo1
Copy link
Owner

stravo1 commented Apr 3, 2024

The recording was probably synced and was renamed to: "Recording....-synced.webm" because as I mentioned earlier attachments are renamed to keep track which are synced and which are not.

The error occurs while trying to show you a pop-up which tells you to rename the attachment in your notes file as well. I have identified the issue and will try and fix it as soon as possible.

fixed: https://github.com/stravo1/obsidian-gdrive-sync/releases/tag/v0.9.9-beta-7-2

@kallekn
Copy link
Author

kallekn commented Apr 4, 2024

Thanks, will test!

@kallekn
Copy link
Author

kallekn commented Apr 4, 2024

Still getting similar error messages. A note that was created on my Android phone seems to have been correctly downloaded to my Mac, but then I get these repeated error messages. Can't see anything odd with the file structure, it's just mydrive -> obsidian -> (name of vault)

Thu Apr 04 2024 16:48:06 GMT+0200 (centraleuropeisk sommartid)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 16:48:11 GMT+0200 (centraleuropeisk sommartid)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 16:48:18 GMT+0200 (centraleuropeisk sommartid)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 16:48:31 GMT+0200 (centraleuropeisk sommartid)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 16:48:33 GMT+0200 (centraleuropeisk sommartid)-Error-FATAL ERROR: Too many errors within a minute.-Error: FATAL ERROR: Too many errors within a minute.
at eval (plugin:obsidian-gdrive-sync:2782:42)

@kallekn
Copy link
Author

kallekn commented Apr 4, 2024

Skärmavbild 2024-04-04 kl  17 00 37

Might this be the problem? In my computer he attachments (imported from Evernote) are in a specific folder called _resources
In Gdrive there is no such folder. Instead, the filename of the attachments begins with _resources/

@stravo1
Copy link
Owner

stravo1 commented Apr 4, 2024

as i mentioned before the plug-in doesn't try to imitate the vault's folder structure on Drive, instead it stores the files with their "relative paths", so the attachments being uploaded as _resources/.... is an expected behaviour.

however this folder might be the cause of the "Folder Already Exists" issue. can you just give me an idea as how the folder structure for the _resources folder might be?

also if you can copy the vault and delete the folder, do you face the same issues? trying doing it with a smaller vault.

i will personally make a folder whose name starts with an underscore (_) to see if there's any issue with folders having such names.

thank you for the info :)

@kallekn
Copy link
Author

kallekn commented Apr 4, 2024

There is no additional structure such as subfolders. Just media files in the folder.

@kallekn
Copy link
Author

kallekn commented Apr 4, 2024

Now I tried to sync my other Mac, with another vault. There is no folder for that vault in the obsidian folder in my Gdrive. Still, the plugin says it is trying to download missing files. Is it looking for files that correspond to the ones already in the vault on my Mac?

I can sync more than one vault at the same time, right?

Then I get this in the error log. Should I just try syncing a vault with no attachments and no folders to see how that works?

Thu Apr 04 2024 20:38:59 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:06 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:13 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:20 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:25 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:31 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:37 GMT+0200 (Keski-Euroopan kesäaika)-Error-Folder already exists.-Error: Folder already exists.
at t. (app://obsidian.md/app.js:1:736472)
at app://obsidian.md/app.js:1:237056
at Object.next (app://obsidian.md/app.js:1:237161)
at a (app://obsidian.md/app.js:1:235879)

Thu Apr 04 2024 20:39:39 GMT+0200 (Keski-Euroopan kesäaika)-Error-FATAL ERROR: Too many errors within a minute.-Error: FATAL ERROR: Too many errors within a minute.
at eval (plugin:obsidian-gdrive-sync:2782:42)

@stravo1
Copy link
Owner

stravo1 commented Apr 4, 2024

if the vault names were the same then the plug-in would assume that you're trying to sync a common vault between two devices and would start downloading files from the folder named after that vault.

yes you should be able to sync multiple vaults, they need to have different names.

thank you for reporting all these errors!
i have been a little busy with my exams and internships, will look into these errors as and when i get time.

@kallekn
Copy link
Author

kallekn commented Apr 8, 2024

It looks like the plugin is trying to sync the same files over and over again. For example, this file exists in one original version and 4 synced versions which seem to have 2 different names. Seemingly both in GDrive and my laptop.

Also I'm getting the same error:
Mon Apr 08 2024 09:10:05 GMT+0200 (centraleuropeisk sommartid)-Error-FATAL ERROR: Too many errors within a minute.-Error: FATAL ERROR: Too many errors within a minute.
at eval (plugin:obsidian-gdrive-sync:2782:42)

Skärmavbild 2024-04-08 kl  10 04 26
Skärmavbild 2024-04-08 kl  10 04 17

@stravo1
Copy link
Owner

stravo1 commented Apr 8, 2024

that's interesting! will look into it soon :)

@stravo1
Copy link
Owner

stravo1 commented Jul 9, 2024

the attachments are renamed when they are uploaded because unlike notes there's no "lastSync" tag that the plug-in can read from the note's content, so to differentiate between which attachments are synced and which are not it's renamed to "attachment_name-synced". i am actively trying to create a workaround for this, but it will take time.

I understand, for new vaults it's an annoyance but for big existing vaults it's a deal breaker. for the This hack works for small vaults that I personally use without too many attachments so I did not see the need to give too much attention to this issue. Now that others are facing this issue, I will try and figure out a proper solution, however it will take time.

attachment issues are now fixed at beta-13

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants