Shorten the cutoff string by using an array in place of a hash

ddnexus · Dec 14, 2024 · dd06c2f · dd06c2f
1 parent 7b37d93
commit dd06c2f
Show file tree

Hide file tree

Showing 11 changed files with 148 additions and 165 deletions.
diff --git a/docs/api/keyset.md b/docs/api/keyset.md
@@ -52,7 +52,7 @@ If you want the best of the two worlds, check out the [keyset_for_ui extra](/doc
 | `set`                      | The `uniquely ordered` `ActiveRecord::Relation` or `Sequel::Dataset` collection to paginate.                                                                                                                                                                                                          |
 | `keyset`                   | The hash of column/direction pairs. Pagy extracts it from the order of the `set`.                                                                                                                                                                                                                     |
 | `keyset attributes`        | The hash of keyset-column/record-value pairs of a record.                                                                                                                                                                                                                                             |
-| `cutoff`                   | A point in the `set` that separates the records of two contiguous `page`s. It's the encoded string of the `keyset attributes` of the last record of a `page`.                                                                                                                                         |
+| `cutoff`                   | A point in the `set` where a `page` ended. Its value is a `Base64` encoded URL-safe string.                                                                                                                                                                                                           |
 | `page`                     | The current `page`, i.e. the page of records beginning after the `cutoff` of the previous page. Also the `:page` variable, which is set to the `cutoff` of the previous page                                                                                                                          |
 | `next`                     | The next `page`, i.e. the page of records beginning after the `cutoff`. Also the `cutoff` value retured by the `next` method.                                                                                                                                                                         |
 
@@ -164,38 +164,32 @@ If you need a specific order:
 
 #### Understanding the Cutoffs
 
-A `cutoff` defines a point in the `set`, right AFTER the last record of a `page`.
+A `cutoff` defines a point in the `set` where a `page` ended. All the records AFTER that point are or will be part of the `next` page.
 
 Let's consider an example of a simple `set`. In order to avoid confusion with numeric ids and number of records, let's assume that
-it that has an `id` column that is actually a unique alphanumeric code, and its order is:
-`order(:id)`.
+it has an `id` column populated by unique alphanumeric codes, and its order is: `order(:id)`.
 
-Assuming a LIMIT of 6, the first page will include the first 6 records in the set: no `cutoff` required so
-far...
+Assuming a LIMIT of 6, the first page will include the first 6 records in the set: no `cutoff` required so far...
 
 ```
             |   page    |                                not yet paginated    |
 beginning ->|. . . . . .|. . . . . . . . . . . . . . . . . . . . . . . . . . .|<- end of set
 ```
 
-After we pull the 6 records, we read the `id` of the last one in the page, which is `F`. So our `cutoff` can be defined like: _"
-the point after the value `F` in the `id` column"_.
+After we pull the first 6 records from the beginning of the `set`, we read the `id` of the last one, which is `F`. So our `cutoff` can be defined like: _"the point up to the value `F` in the `id` column"_.
 
-Notice that this is not like saying _"after the record `F`"_. It's important to understand that a `cutoff` refers just to a value
-in a column (or multiple column in case of muti-columns keysets) after which there will be the next page.
-
-Indeed, that very record could be deleted right after we read it, and our `cutoff` will still be the valid start for the next page
-of 6 records after the `cutoff-F`...
+Notice that this is not like saying _"up to the record `F`"_. It's important to understand that a `cutoff` refers just to a value
+in a column (or a combination of multiple column, in case of muti-columns keysets).
 
+Indeed, that very record could be deleted right after we read it, and our `cutoff` will still be the valid reference that _"we paginated the `set`, up to the "F" value"_...
 ```
             |   page    |   page    |                    not yet paginated    |
 beginning ->|. . . . . F]. . . . . .|. . . . . . . . . . . . . . . . . . . . .|<- end of set
                         |
                      cutoff-F
 ```
 
-Again, after we pull the next 6 records, we read the `id` of the last one in the page, which is `L`: so we have our new
-`cutoff-L`, which is the start of the next `page`...
+For getting the `next` page of records - this time - we pull the `next` 6 records AFTER the `cutoff-F`. Again, we read the `id` of the last one, which is `L`: so we have our new `cutoff-L`, which is the end of the current `page`, and the `next` will go AFTER it...
 
 ```
             |   page    |   page    |   page    |        not yet paginated    |
@@ -204,7 +198,7 @@ beginning ->|. . . . . F]. . . . . L]. . . . . .|. . . . . . . . . . . . . . .|<
                      cutoff-F    cutoff-L
 ```
 
-Pagy encodes the values of the `cutoffs` in a `Base64` URL-safe string that is used in the request as a param.
+Pagy encodes the values of the `cutoffs` in a `Base64` URL-safe string that is sent as a param in the `request`.
 
 ## ORMs
 
@@ -254,7 +248,7 @@ Default `nil`.
 
 ==- `:jsonify_keyset_attributes`
 
-A lambda to override the generic json encoding of the `keyset` attributes. Use it when the generic `to_json` method would lose
+A lambda to override the generic json encoding of the `keyset` attributes. It receives the keyset attributes to jsonify, and it should return a JSON string of the `attributes.values` array. Use it when the generic `to_json` method would lose
 some information when decoded.
 
 For example: `Time` objects may lose or round the fractional seconds through the encoding/decoding cycle, causing the ordering to
@@ -266,7 +260,7 @@ etc.). Here is what you can do:
 jsonify_keyset_attributes = lambda do |attributes|
   # Convert it to a string matching the stored value/format in SQLite DB
   attributes[:created_at] = attributes[:created_at].strftime('%F %T.%6N')
-  attributes.to_json
+  attributes.values.to_json  # remember to return an array of the values only
 end
 
 Pagy::Keyset(set, jsonify_keyset_attributes:)

diff --git a/docs/api/keyset_for_ui.md b/docs/api/keyset_for_ui.md
@@ -33,11 +33,11 @@ You should also familiarize with the [Pagy::Keyset](keyset.md) class.
 
 This section integrates the [Keyset Glossary](keyset_for_ui.md#glossary)
 
-| Term                 | Description                                                                                                                                                                                   |
-|----------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Term                       | Description                                                                                                                                                                                   |
+|----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | `keyset pagination for UI` | The pagy exclusive technique to use `keyset pagination` with numeric pages, supporting `pagy_*navs` and the other Frontend helpers.<br/>The best technique for performance AND functionality! |
-| `page`               | The current page **number**                                                                                                                                                                   |
-| `cutoffs`            | The `cutoff`s of the pagination known so far, used to keep track of the visited pages.                                                                                                           |
+| `page`                     | The current page **number**                                                                                                                                                                   |
+| `cutoffs`                  | The `cutoff`s of the known pagination state, used to keep track of the visited pages during the navigation.                                                                                   |
 
 ## How Pagy Keyset For UI works
 

diff --git a/docs/extras/keyset_for_ui.md b/docs/extras/keyset_for_ui.md
@@ -17,11 +17,10 @@ and the other Frontend helpers.
 
 ## Overview
 
-This extra manages the cache used by the `Pagy::KeysetForUI` instance, allowing easy customization and integration with your
-app.
+This extra manages the cache used by the `Pagy::KeysetForUI` instance, allowing easy customization and integration with your app.
 
-It also adds a `pagy_keyset_for_ui` constructor method that can be used in your controllers, and provides the automatic setting
-of the variables from the request `params`.
+It also adds a `pagy_keyset_for_ui` constructor method that can be used in your controllers, and provides the automatic setting of
+the variables from the request `params`.
 
 Please refer to the following resource:
 
@@ -53,8 +52,8 @@ def pagy_cache_new_key = my_custom_cache.generate_key
 
 ## Understanding the cache
 
-This extra uses the `session` object as the cache for the `cutoffs` (not the records!) by default, because it's simple and works in any app, at least for
-prototyping.
+This extra uses the `session` object as the cache for the `cutoffs` (not for the records!) by default, because it's simple and
+works in any app, at least for prototyping.
 
 Notice that the `cutoffs` array can potentially grow big if you don't use `:max_pages`, especially if your `keyset` contains
 multiple ordered columns and more if their size is big. You must be aware of it.
@@ -69,8 +68,8 @@ session as the cache (e.g. `ActiveRecord::SessionStore`).
 
 !!!warning
 
-Besides writing and reading from it, Pagy does not expire nor handle the cache in any way. Your app should manage it (e.g. like it does
-with the `session` object).
+Besides writing and reading from it, Pagy does not expire nor handle the cache in any way. Your app should manage it (e.g. like it
+does with the `session` object).
 !!!
 
 This extra uses only 3 simple methods to handle the cache:
@@ -86,7 +85,8 @@ handling other aspects of it (e.g. expiration, etc.)
 
 We are considering implementing a client-side cache using the Browser's `sessionStorage`.
 
-It might considerably simplify the handling of the cache, but it will require some time to design it properly, so please, hang tight and cheer for us!
+It might considerably simplify the handling of the cache, but it will require some time to design it properly, so please, hang
+tight and cheer for us!
 !!!
 
 ## Variables

diff --git a/gem/lib/pagy/keyset.rb b/gem/lib/pagy/keyset.rb
@@ -105,10 +105,8 @@ def after_cutoff_sql(prefix = nil)
 
     # Decode a cutoff, check its consistency and returns the cutoff args
     def cutoff_to_args(cutoff)
-      args = JSON.parse(B64.urlsafe_decode(cutoff)).transform_keys(&:to_sym)
-      raise InternalError, 'cutoff and keyset are not consistent' \
-            unless args.keys == @keyset.keys
-
+      values = JSON.parse(B64.urlsafe_decode(cutoff))
+      args   = @keyset.keys.zip(values).to_h
       typecast_args(args)
     end
 
@@ -123,8 +121,8 @@ def default
 
     # Derive the cutoff from the last record
     def derive_cutoff
-      hash = keyset_attributes_from(@records.last)
-      json = @vars[:jsonify_keyset_attributes]&.(hash) || hash.to_json
+      attr = keyset_attributes_from(@records.last)
+      json = @vars[:jsonify_keyset_attributes]&.(attr) || attr.values.to_json
       B64.urlsafe_encode(json)
     end
 

diff --git a/test/pagy/extras/headers_test.rb b/test/pagy/extras/headers_test.rb
@@ -79,10 +79,13 @@
     it 'returns custom headers hash' do
       pagy, _records = app.send(:pagy_keyset,
                                 Pet.order(:id),
-                                page: 'eyJpZCI6MjB9',
+                                page: 'WzIwXQ',
                                 headers: { limit: 'Per-Page', page: 'Page', count: 'Total', pages: false })
       _(app.send(:pagy_headers, pagy)).must_rematch :headers
     end
+    # -{"link"=>"<http://example.com:3000/foo?a=one&b=two&page>; rel=\"first\", <http://example.com:3000/foo?a=one&b=two&page=eyJpZCI6NDB9>; rel=\"next\"", "Page"=>"eyJpZCI6MjB9", "Per-Page"=>"20"}
+    # +{"link"=>"<http://example.com:3000/foo?a=one&b=two&page>; rel=\"first\"", "Page"=>"eyJpZCI6MjB9", "Per-Page"=>"20"}
+
     it 'omit next on last page' do
       pagy, _records = app.send(:pagy_keyset, Pet.order(:id), limit: 50)
       _(app.send(:pagy_headers, pagy)).must_rematch :headers