Skip to content

Execute less Redis commands while dequeuing #51

@oli-h

Description

@oli-h

Situation

RedisQues executes the following sequence of eight Redis commands for dequeue of one message.
Example with a queue named "oli":

1531139048.907426 "GET"     "redisques:consumers:oli"
1531139048.907666 "EXPIRE"  "redisques:consumers:oli" "20"
1531139048.907805 "GET"     "redisques:consumers:oli"
1531139048.907939 "HEXISTS" "redisques:locks" "oli"
1531139048.908095 "LINDEX"  "redisques:queues:oli" "0"
1531139048.908418 "ZADD"    "redisques:queues" "1.531139048908E12" "oli"
1531139048.909548 "LPOP"    "redisques:queues:oli"
1531139048.909864 "LLEN"    "redisques:queues:oli"

Most of them are only executed after the previous one's response is received. RedisQues therefor suffers more from higher roundtrip times towards Redis server.

Some ideas to improve this

  • Remove the call to EXPIRE redisques:consumers:oli 20 from the dequeuing sequence. RedisQues already refreshes consumer registration independently with a timer. No need to repeat this while dequeuing.
  • Remove the seconds GET redisques:consumers:oli - Method RedisQues.consume(String queue) is only called for the current registered consumer - no need to check this here again
  • The LPOP redisques:queues:oli is only used to remove the head-of-queue after successful message delivery - Redis response is not used. Better use LTRIM ...queue 1 -1 to remove head-of-queue without returning the message content.
  • Don't call LLEN redisques:queues:oli to check if queue is empty before starting the next consumption. Just do it, avoid this extra Redis command. Later, the LINDEX ... will detect the empty queue and stop processing
  • Do we really need to lookup 'who is' the consumer of a queue while 'we' are consuming this queue at the moment? In this case would easily assume that 'we' are still the consumer - remember that 'we' have a timer which refreshes that 'we' are the consumer.
    In other words: No need for GET redisques:consumers:oli while we are in the consumption race. Perhaps we can have a look at myQueues to cross-check if we're still the consumer of this queue

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions