Using :keyset returns Array

[kjakub]

@pagy, @users = pagy(:keyset, User.order(:id))

returns Array, would it be possible to retain ActiveRecord Relation ?

[ddnexus]

No. We need to retrieve the keyset values of the last record in the page to provide the next page and URL. An AR Relation is just representing the query, not the actual data.

[kjakub]

in my particular case down in controller return unless stale?(strong_etag: @users.cache_key_with_version) and yes hard time to match stong etag - since - cache_key_with_version has its own pitfal with rails/rails#37724 - i was able to patch it (compute md5 in postgres technique) - but thats another topic

not sure what you mean by - DB is not static, but i hope i gave okeyish example why AR Relation might be still needed

[ddnexus]

etag_value = Digest::XXH3_64bits.hexdigest(records.map(&:cache_key_with_version).join)
response.headers['ETag'] = %Q("#{etag_value}")

I use xxh in another gem holdify, and it's super fast and produces very short keys.

However, this is NOT a solution, because it requires to EXECUTE a query in order to avoid EXECUTING a query! 🙃

[ddnexus]

Oh... sorry I missed your return unless stale?. Got it. (I edited my previous answer)

However if you were used to offset pagination, you might not need caching with keyset. it's really really fast and light... you may want to do some benchmark.

BTW, the eager-loading of :keyset defeat the reason to use strong etag (which is avoiding the DB query), however it also makes it so fast and light that it will solve the problem at the source.

[kjakub]

• aha! Somehow similar approach my colleague suggested, but i was benchmarking it on large dataset and it was the slowest strategy (obviously). But nevermind, i feel like i went to far so we can limit the page with lower value, so it won't be notiacable.

• good point with no-etag approach, since it will be way faster now

• I am also thinking maybe when have time i could look into the your pagy gem and see it from inside, train my brain and perhaps .load the AR relation and retain it

Thanks a lot

[ddnexus]

The bigger the dataset the better the keyset gain.

I know a couple of big companies use keyset with almost billion (with a B) records and multicolumn keysets, and they don't cache.

Just ensure to have the right (b-tree) index, and it will pull your records directly with an index seek, never hitting a full table scan.