-
-
Notifications
You must be signed in to change notification settings - Fork 139
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/delete duplicates #301
base: master
Are you sure you want to change the base?
Changes from all commits
a9a6e4b
120b475
b94372b
03d4a9c
5807459
96a47c6
0b737ad
4142dab
2bd2060
d95f917
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -80,6 +80,20 @@ def delete_directory_if_empty(self, directory_path): | |
|
||
return False | ||
|
||
def delete_file(self, file_path): | ||
"""Delete a file safely but permanently. | ||
""" | ||
try: | ||
if os.path.exists(file_path): | ||
os.remove(file_path) | ||
return True | ||
else: | ||
print("The file does not exist.") | ||
except OSError: | ||
pass | ||
|
||
return False | ||
|
||
def get_all_files(self, path, extensions=None): | ||
"""Recursively get all files which match a path and extension. | ||
|
||
|
@@ -508,10 +522,17 @@ def process_file(self, _file, destination, media, **kwargs): | |
checksum_file = db.get_hash(checksum) | ||
if(allow_duplicate is False and checksum_file is not None): | ||
if(os.path.isfile(checksum_file)): | ||
log.info('%s already exists at %s. Skipping...' % ( | ||
_file, | ||
checksum_file | ||
)) | ||
if constants.delete_duplicates: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Need to import constants at the top of the file. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would rather include this check in |
||
log.info('%s already exists at %s. Deleting...' % ( | ||
_file, | ||
checksum_file | ||
)) | ||
self.delete_file(_file) | ||
else: | ||
log.info('%s already exists at %s. Skipping...' % ( | ||
_file, | ||
checksum_file | ||
)) | ||
return | ||
else: | ||
log.info('%s matched checksum but file not found at %s. Importing again...' % ( # noqa | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should add tests for this including for when
delete_duplicates
isTrue
andFalse
.