Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Original checksum returned None for multiple files. #404

Open
fredrike opened this issue May 4, 2021 · 4 comments · May be fixed by #417
Open

Original checksum returned None for multiple files. #404

fredrike opened this issue May 4, 2021 · 4 comments · May be fixed by #417

Comments

@fredrike
Copy link

fredrike commented May 4, 2021

Hi,

Great tool and very useful!

I'm getting this error for a lot of my files from Google Take Out:

Original checksum returned None for /source/gto/takeout/fer/Takeout/Google Photos/Familj och vänner/IMG_5683.JPG. Skipping...

Just to test I extracted the parts that does the checksum and ran it manually:

# python3 filetest.py import --file "/source/gto/takeout/fer/Takeout/Google Photos/Familj och vänner/IMG_5683.JPG"
pass
/source/gto/takeout/fer/Takeout/Google Photos/Familj och vänner/IMG_5683.JPG
/source/gto/takeout/fer/Takeout/Google Photos/Familj och vänner/IMG_5683.JPG
9ca92b26f96e9aace58420b4dca2d0a13f3aba55b5c4f8331aead18bcbbfd18b
9ca92b26f96e9aace58420b4dca2d0a13f3aba55b5c4f8331aead18bcbbfd18b

/elodie # cat filetest.py

import sys
import hashlib
import click

def _decode(string, encoding=sys.getfilesystemencoding()):
    """Return a utf8 encoded unicode string.
    Python2 and Python3 differ in how they handle strings.
    So we do a few checks to see if the string is ascii or unicode.
    Then we decode it if needed.
    """
    if hasattr(string, 'decode'):
        # If the string is already unicode we return it.
        try:
            if isinstance(string, unicode):
                return string
        except NameError:
            pass

        return string.decode(encoding)

    return string


def checksum(file_path, blocksize=65536):
    """Create a hash value for the given file.
    See http://stackoverflow.com/a/3431835/1318758.
    :param str file_path: Path to the file to create a hash for.
    :param int blocksize: Read blocks of this size from the file when
        creating the hash.
    :returns: str or None
    """
    hasher = hashlib.sha256()
    with open(file_path, 'rb') as file:
        buf = file.read(blocksize)

        while len(buf) > 0:
            hasher.update(buf)
            buf = file.read(blocksize)
        return hasher.hexdigest()
    return None

@click.command('import')
@click.option('--file', type=click.Path(dir_okay=False),
              help='Import this file, if specified.')
def _import(file):
    print(file)
    _file = _decode(file)
    print(_file)
    print(checksum(file))
    print(checksum(_file))

@click.group()
def main():
    print("pass")


main.add_command(_import)



if __name__ == '__main__':
    main()
@fredrike
Copy link
Author

fredrike commented May 5, 2021

Perhaps this is just a debug issue, while reviewing the log (had to re-run elodie on my ~250'000 files) I saw this:

/source/photos/Bilder/Peru/P1010097.JPG already at /destination/2009-03-Mar/Unknown Location,/2009-03-16_10-27-50 dmc-lx3 p1010097.jpg.
Original checksum returned None for /source/photos/Bilder/Peru/P1010097.JPG. Skipping...
/source/photos/Pictures/Rebuilt Library/Masters/2013/02/13/20130213-053230/P1000291.JPG already at /destination/2012-02-Feb/Unknown Location,/2012-02-10_11-32-50 dmc-lx7 p1000291.jpg.
Original checksum returned None for /source/photos/Pictures/Rebuilt Library/Masters/2013/02/13/20130213-053230/P1000291.JPG. Skipping...

So it seems that the message perhaps should be something different.

@fredrike
Copy link
Author

what is the status on this?

@fredrike fredrike linked a pull request Jul 31, 2022 that will close this issue
@iraklis10
Copy link

I get the same error message

@gilles6
Copy link

gilles6 commented Jul 5, 2023

Same error, same question, any explanation on this error?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants