Merge branch 'develop'

docs/developer_api.md

@@ -36,7 +36,7 @@ In general, the API deals with standard UTF-8 JSON. POST requests and 200 OK res
 *   In python, converting your tag list to the URL encoded string would be:
 
     ```
-    urllib.parse.quote( json.dumps( tag_list ) )
+    urllib.parse.quote( json.dumps( tag_list ), safe = '' )
     ```
 
 *   Full URL path example:

docs/old_changelog.html

@@ -34,6 +34,50 @@
         <div class="content">
             <h1 id="changelog"><a href="#changelog">changelog</a></h1>
             <ul>
+                <li>
+                    <h2 id="version_570"><a href="#version_570">version 570</a></h2>
+                    <ul>
+                        <li><h3>UI stuff</h3></li>
+                        <li>wrote a thing to wrap tooltips and applied it everywhere. every tooltip in the program should now wrap to 80 characters</li>
+                        <li>the thumbnail view is now better about pausing the current video if you open it externally in various ways</li>
+                        <li>the 'open' submenu you get off of a file right-click is now exactly the same for the thumbnail menu and the media viewer menu, with all commands working in either place, the labels are also brushed up a little</li>
+                        <li>added a shortcut action for 'open file in web browser' to the media shortcut set</li>
+                        <li>added a shortcut action for 'open files in a new duplicates filter page' to the media shortcut set</li>
+                        <li>added/updated the shortcut action for 'open similar looking files in a new page' in the media shortcut set. this is now one job that lets you set any distance, and it now works from the media viewer too. all existing `show similar files: 0 (exact)` fixed-distance simple actions will be converted to the new action when you update</li>
+                        <li>I removed 'open externally' and 'open in file explorer' shortcuts from the media viewer/preview viewer/thumbnails sets. these sets are technically awkward and were really meant for a different thing, like pause/play or 'close media viewer', and having the media command code duplicated here was getting spammy. if you have any of these now-defunct commands set, please move them up to the general 'media' set, where it'll work everywhere. sorry if this breaks a very complicated set you have, but let's KISS!</li>
+                        <li>the 'files' submenu off thumbnails or the media viewer is flattened one level. the 'upload to' remote services stuff still isn't available for the media viewer, but I'll do the same as I did above for that in the near future</li>
+                        <li><h3>misc</h3></li>
+                        <li>fixed an issue with the 'manage tag siblings/parents' dialogs where the mass-import button was, in 'add or delete' mode, not doing any deletes/rescinds if there were any new pairs in what was being imported. this was probably applying to large regular adds in the UI, also</li>
+                        <li>this mass-import button of 'manage tag siblings/parents' also dedupes the pairs coming in. it now shouldn't do anything like 'add, then ask to remove' if you have the same pair twice!</li>
+                        <li>the nitter downloaders are removed from the defaults. I can't keep up with whatever the situation is there</li>
+                        <li>the style and stylesheet names in the options are now sorted</li>
+                        <li>sidecar importers will now work on sidecars that have uppercase .TXT or .JSON extensions</li>
+                        <li><h3>more URL stuff (advanced, can be ignored by most)</h3></li>
+                        <li>fixed up the recent URL encoding tech to properly follow the encoding exceptions as under RFC 3986. an '@' in an URL shouldn't get messed up now. thanks to the user(s) who helped here</li>
+                        <li>incoming URLs can now have a mix of encoded and non-encoded characters and the 'ensure URL is encoded' process will accept it and encode the non-encoded parts, idempotently. it only fails on ingesting a legit decoded percent character that happens to be followed by two hex chars, but that's rare enough we don't really have to worry</li>
+                        <li>you can similarly now enter multiple tags in a query text that are a mix of encoded and non-encoded, a mix of %20 and spaces, and it should figure it out</li>
+                        <li>the 'ensure URL is encoded' process now applies to GUG-generated URLs, and in the edit GUG UI, you now see the normalised 'for server' URL, with any additional tokens or whatever the URL class has</li>
+                        <li>GUGs also try to recognise if their replacement phrase is going into the path or the parameters now, and only force-encodes everything if it looks like our tags are going into a query param</li>
+                        <li>ensured that what you paste into an 'edit URL Class' panel's 'example url' section gets encoded before normalisation just as it would in engine</li>
+                        <li>the file log right-click now shows both the normalised and request urls under the 'additional urls' section, if they differ from the pretty human URL in the list</li>
+                        <li>right-clicking a single item in the downloader search log now previews the specific request URL to be copied</li>
+                        <li><h3>boring stuff</h3></li>
+                        <li>all instances of URL path or parameter encoding now go through one location that obeys RFC 3986</li>
+                        <li>replaced my various uses of the unusual `ParseResult` with `urllib.parse.urlunparse`</li>
+                        <li>added a couple unit tests for the improved URL encoding tech</li>
+                        <li>added some unit tests for the GUGs' new encoding tech</li>
+                        <li>harmonised how a file is opened in the OS file explorer in the media results and media canvas pages. what was previously random hardcode, duplicated internal method calls, and ancient pubsub redirects now all goes thorugh the application command system to a singular isolated media-actioning method</li>
+                        <li>did the same harmonisation for opening files externally</li>
+                        <li>and for opening files in your web browser, which gets additional new infrastructure so it can plug into the shortcuts system</li>
+                        <li>and to a lesser degree the 'open in a new page' and 'open in a new duplicates filter page' commands</li>
+                        <li>moved the various gui-side media python files to a new 'gui.media' module. renamed `ClientGUIMedia` to `ClientGUISimpleActions` and `ClientGUIMediaActions` to `ClientGUIModalActions` and shuffled their methods back and forth a bit</li>
+                        <li>cleaned up `ClientGUIFunctions` and `ClientGUICommon` and their imports a little with some similar shuffle-refactoring</li>
+                        <li>broke up `ClientGUIControls` into a bunch of smaller, defined files, mostly to untangle imports</li>
+                        <li>cleaned up how some text and exceptions are split by newlines to handle different sorts of newline, and cleaned up how I fetch the first 'summary' line of text in all cases across the program</li>
+                        <li>replaced `os.linesep` with `\n` across the program. Qt only wants `\n` anyway, most logging wants `\n` (and sometimes converts on the fly behind the scenes), and this helps KISS otherwise. I might bring back `os.linesep` for sidecars and stuff if it proves a problem, but most text editors and scripting languages are very happy with `\n`, so we'll see</li>
+                        <li>multi-column lists now show multiline tooltips if the underlying text in the cell was originally multiline (although tbh this is rare)</li>
+                    </ul>
+                </li>
                 <li>
                     <h2 id="version_569"><a href="#version_569">version 569</a></h2>
                     <ul>

hydrus/client/ClientApplicationCommand.py

@@ -6,6 +6,7 @@
 from hydrus.core import HydrusSerialisable
 from hydrus.core import HydrusTime
 
+from hydrus.client import ClientConstants as CC
 from hydrus.client import ClientGlobals as CG
 
 SIMPLE_ARCHIVE_DELETE_FILTER_BACK = 0
@@ -161,6 +162,9 @@
 SIMPLE_REARRANGE_THUMBNAILS = 150
 SIMPLE_MAC_QUICKLOOK = 151
 SIMPLE_COPY_URLS = 152
+SIMPLE_OPEN_FILE_IN_WEB_BROWSER = 153
+SIMPLE_OPEN_SELECTION_IN_NEW_DUPLICATES_FILTER_PAGE = 154
+SIMPLE_OPEN_SIMILAR_LOOKING_FILES = 155
 
 REARRANGE_THUMBNAILS_TYPE_FIXED = 0
 REARRANGE_THUMBNAILS_TYPE_COMMAND = 1
@@ -265,8 +269,11 @@
     SIMPLE_NEW_WATCHER_DOWNLOADER_PAGE : 'open a new page: thread watcher',
     SIMPLE_OPEN_FILE_IN_EXTERNAL_PROGRAM : 'open file in external program',
     SIMPLE_OPEN_FILE_IN_FILE_EXPLORER : 'open file in file explorer',
+    SIMPLE_OPEN_FILE_IN_WEB_BROWSER : 'open file in web browser',
     SIMPLE_OPEN_KNOWN_URL : 'open known url',
     SIMPLE_OPEN_SELECTION_IN_NEW_PAGE : 'open files in a new page',
+    SIMPLE_OPEN_SELECTION_IN_NEW_DUPLICATES_FILTER_PAGE : 'open files in a new duplicates filter page',
+    SIMPLE_OPEN_SIMILAR_LOOKING_FILES : 'open similar looking files in a new page',
     SIMPLE_PAN_BOTTOM_EDGE : 'pan file to bottom edge',
     SIMPLE_PAN_DOWN : 'pan file down',
     SIMPLE_PAN_HORIZONTAL_CENTER : 'pan file left/right to center',
@@ -465,7 +472,7 @@ class ApplicationCommand( HydrusSerialisable.SerialisableBase ):
 
     SERIALISABLE_TYPE = HydrusSerialisable.SERIALISABLE_TYPE_APPLICATION_COMMAND
     SERIALISABLE_NAME = 'Application Command'
-    SERIALISABLE_VERSION = 5
+    SERIALISABLE_VERSION = 6
 
     def __init__( self, command_type = None, data = None ):
 
@@ -656,6 +663,49 @@ def _UpdateSerialisableInfo( self, version, old_serialisable_info ):
             return ( 5, new_serialisable_info )
 
 
+        if version == 5:
+
+            ( command_type, serialisable_data ) = old_serialisable_info
+
+            if command_type == APPLICATION_COMMAND_TYPE_SIMPLE:
+
+                data_dict = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_data )
+
+                simple_action = data_dict[ 'simple_action' ]
+
+                if simple_action in ( SIMPLE_GET_SIMILAR_TO_EXACT, SIMPLE_GET_SIMILAR_TO_VERY_SIMILAR, SIMPLE_GET_SIMILAR_TO_SIMILAR, SIMPLE_GET_SIMILAR_TO_SPECULATIVE ):
+
+                    hamming_distance = 0
+
+                    if simple_action == SIMPLE_GET_SIMILAR_TO_EXACT:
+
+                        hamming_distance = 0
+
+                    elif simple_action == SIMPLE_GET_SIMILAR_TO_VERY_SIMILAR:
+
+                        hamming_distance = 2
+
+                    elif simple_action == SIMPLE_GET_SIMILAR_TO_SIMILAR:
+
+                        hamming_distance = 4
+
+                    elif simple_action == SIMPLE_GET_SIMILAR_TO_SPECULATIVE:
+
+                        hamming_distance = 8
+
+
+                    data_dict[ 'simple_action' ] = SIMPLE_OPEN_SIMILAR_LOOKING_FILES
+                    data_dict[ 'simple_data' ] = hamming_distance
+
+
+                serialisable_data = data_dict.GetSerialisableTuple()
+
+
+            new_serialisable_info = ( command_type, serialisable_data )
+
+            return ( 6, new_serialisable_info )
+
+
 
     def GetCommandType( self ):
 
@@ -756,6 +806,19 @@ def ToString( self ):
 
                 s = f'{s} ({direction_s} {ms_s})'
 
+            elif action == SIMPLE_OPEN_SIMILAR_LOOKING_FILES:
+
+                hamming_distance = self.GetSimpleData()
+
+                if hamming_distance in CC.hamming_string_lookup:
+
+                    s = f'{s} ({hamming_distance} - {CC.hamming_string_lookup[ hamming_distance ]})'
+
+                else:
+
+                    s = f'{s} ({hamming_distance})'
+
+
             elif action == SIMPLE_MOVE_THUMBNAIL_FOCUS:
 
                 ( move_direction, selection_status ) = self.GetSimpleData()

hydrus/client/ClientController.py

@@ -566,7 +566,7 @@ def qt_code():
                 from hydrus.client.gui import ClientGUIDialogsQuick
 
                 message = 'It looks like another instance of this client is already running, so this instance cannot start.'
-                message += os.linesep * 2
+                message += '\n' * 2
                 message += 'If the old instance is closing and does not quit for a _very_ long time, it is usually safe to force-close it from task manager.'
 
                 result = ClientGUIDialogsQuick.GetYesNo( self._splash, message, title = 'The client is already running.', yes_label = 'wait a bit, then try again', no_label = 'forget it' )
@@ -1686,9 +1686,9 @@ def RestoreDatabase( self ):
                 path = dlg.GetPath()
 
                 text = 'Are you sure you want to restore a backup from "{}"?'.format( path )
-                text += os.linesep * 2
+                text += '\n' * 2
                 text += 'Everything in your current database will be deleted!'
-                text += os.linesep * 2
+                text += '\n' * 2
                 text += 'The gui will shut down, and then it will take a while to complete the restore. Once it is done, the client will restart.'
 
                 result = ClientGUIDialogsQuick.GetYesNo( self.gui, text )

hydrus/client/ClientDBMaintenanceManager.py

@@ -155,7 +155,7 @@ def check_shutdown():
                         HydrusData.PrintException( e )
 
                         message = 'There was an unexpected problem during deferred table delete database maintenance work! This maintenance system will not run again this boot. A full traceback of this error should be written to the log.'
-                        message += os.linesep * 2
+                        message += '\n' * 2
                         message += str( e )
 
                         HydrusData.ShowText( message )

hydrus/client/ClientData.py

@@ -30,15 +30,15 @@ def CatchExceptionClient( etype, value, tb ):
 
         pretty_value = str( value )
 
-        if os.linesep in pretty_value:
-            
-            ( first_line, anything_else ) = pretty_value.split( os.linesep, 1 )
-            
-            trace = trace + os.linesep + anything_else
+        all_lines = pretty_value.splitlines()
+
+        first_line = all_lines[0]
+
+        if len( all_lines ) > 1:
 
-        else:
+            the_rest = all_lines[1:]
 
-            first_line = pretty_value
+            trace = trace + '\n' + '\n'.join( the_rest )
 
 
         job_status = ClientThreading.JobStatus()
@@ -68,7 +68,7 @@ def CatchExceptionClient( etype, value, tb ):
 
         text = 'Encountered an error I could not parse:'
 
-        text += os.linesep
+        text += '\n'
 
         text += str( ( etype, value, tb ) )
 
@@ -153,7 +153,7 @@ def ShowExceptionTupleClient( etype, value, tb, do_wait = True ):
 
     if tb is None:
 
-        trace = 'No error trace--here is the stack:' + os.linesep + ''.join( traceback.format_stack() )
+        trace = 'No error trace--here is the stack:' + '\n' + ''.join( traceback.format_stack() )
 
     else:
 
@@ -162,15 +162,15 @@ def ShowExceptionTupleClient( etype, value, tb, do_wait = True ):
 
     pretty_value = str( value )
 
-    if os.linesep in pretty_value:
-        
-        ( first_line, anything_else ) = pretty_value.split( os.linesep, 1 )
-        
-        trace = trace + os.linesep + anything_else
+    all_lines = pretty_value.splitlines()
+
+    first_line = all_lines[0]
+
+    if len( all_lines ) > 1:
 
-    else:
+        the_rest = all_lines[1:]
 
-        first_line = pretty_value
+        trace = trace + '\n' + '\n'.join( the_rest )
 
 
     job_status = ClientThreading.JobStatus()