Skip to content

fix flaky integration tests#3720

Merged
tishun merged 4 commits intoredis:mainfrom
viktoriya-kutsarova:fix/flaky-integration-tests
Apr 16, 2026
Merged

fix flaky integration tests#3720
tishun merged 4 commits intoredis:mainfrom
viktoriya-kutsarova:fix/flaky-integration-tests

Conversation

@viktoriya-kutsarova
Copy link
Copy Markdown
Contributor

@viktoriya-kutsarova viktoriya-kutsarova commented Apr 15, 2026

As a continuation of #3724:

Fix flaky and broken integration tests

  • Makefile: pass TEST_WORK_FOLDER to test targets. Locally, make start places TLS certs under work/ but TlsSettings defaults TEST_WORK_FOLDER to work/docker/. Added TEST_WORK_FOLDER=$(REDIS_ENV_WORK_DIR) to make test and make test-coverage to align with the CI pipeline.
  • SslIntegrationTests: move assumeTrue before truststore creation. The connectivity check was placed after createAndSaveTestTruststore calls, causing a RuntimeException instead of a graceful skip when stunnel is unavailable.
  • SslIntegrationTests: fix wrong truststore in standaloneWithOpenSsl. The test used truststoreFile0 (redis-standalone-0) but connects to port 6443 which is served by redis-standalone-1. Changed to truststoreFile1 to match standaloneWithJdkSsl.
  • KeyCommandIntegrationTests: fix clock-skew flakiness in pexpireat. PEXPIREAT uses an absolute JVM timestamp, so PTTL can exceed the expected ceiling when the Redis server clock is slightly behind. Replaced the hardcoded upper bound with a dynamic calculation plus a clockSkewToleranceMs tolerance.
  • KeyCommandIntegrationTests: fix clock-skew flakiness in restoreReplace. Same issue with RestoreArgs.absttl() — applied the same dynamic upper bound fix. Aligned expiration to 5 seconds for consistency with pexpireat.
  • HotkeysCommandIntegrationTests: fix unreliable key capture in hotkeysStartOptions. The DURATION test section used the default sample ratio, which could skip the single SET command. Added sample(1) to ensure every command is captured, consistent with hotkeysBothMetrics.
  • RedisVectorSetIntegrationTests: use isCloseTo for similarity scores. Vector similarity scores can vary slightly across Redis versions and platforms. Replaced exact isEqualTo assertions with isCloseTo(value, within(0.001)), consistent with RedisVectorSetAdvancedIntegrationTests.
  • Fixed the following flake in the RedisClusterPubSubConnectionIntegrationTests
[INFO] Running io.lettuce.core.cluster.pubsub.RedisClusterPubSubConnectionIntegrationTests
Error:  Tests run: 23, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.634 s <<< FAILURE! -- in io.lettuce.core.cluster.pubsub.RedisClusterPubSubConnectionIntegrationTests
Error:  io.lettuce.core.cluster.pubsub.RedisClusterPubSubConnectionIntegrationTests.testConnectToLeastClientsNode -- Time elapsed: 0.021 s <<< FAILURE!
java.lang.AssertionError: 

Expecting actual:
  "1c541b6daf98719769e6aacf338a7d81f108a180"
not to be equal to:
  "1c541b6daf98719769e6aacf338a7d81f108a180"

	at io.lettuce.core.cluster.pubsub.RedisClusterPubSubConnectionIntegrationTests.testConnectToLeastClientsNode(RedisClusterPubSubConnectionIntegrationTests.java:297)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at java.util.ArrayList.forEach(ArrayList.java:1259)
	at java.util.ArrayList.forEach(ArrayList.java:1259)
  • removed the log pollution caused by the StatefulMultiDbConnectionIntegrationTests
INFO] Running io.lettuce.core.failover.StatefulMultiDbConnectionIntegrationTests
Error: Exception in thread "Thread-15" io.lettuce.core.RedisConnectionException: Unable to connect
	at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:79)
	at io.lettuce.core.failover.StatefulRedisMultiDbConnectionImpl.lambda$addDatabase$26(StatefulRedisMultiDbConnectionImpl.java:1066)
	at io.lettuce.core.failover.StatefulRedisMultiDbConnectionImpl.doByExclusiveLock(StatefulRedisMultiDbConnectionImpl.java:965)
	at io.lettuce.core.failover.StatefulRedisMultiDbConnectionImpl.addDatabase(StatefulRedisMultiDbConnectionImpl.java:1045)
	at io.lettuce.core.failover.StatefulRedisMultiDbConnectionImpl.addDatabase(StatefulRedisMultiDbConnectionImpl.java:1023)
	at io.lettuce.core.failover.StatefulMultiDbConnectionIntegrationTests.lambda$shouldHandleConcurrentAddsAndRemovesOnMultipleUris$13(StatefulMultiDbConnectionIntegrationTests.java:629)
	at java.lang.Thread.run(Thread.java:750)

Make sure that:

  • You have read the contribution guidelines.
  • You have created a feature request first to discuss your contribution intent. Please reference the feature request ticket number in the pull request.
  • You applied code formatting rules using the mvn formatter:format target. Don’t submit any formatting related changes.
  • You submit test cases (unit or integration tests) that back your changes.

Note

Low Risk
Changes are limited to build/test harness and integration test assertions; main risk is accidentally masking real regressions by making expectations more permissive.

Overview
Reduces integration test flakiness by aligning make test/make test-coverage with the docker environment work directory (exporting TEST_WORK_FOLDER) so TLS artifacts are discovered consistently.

Fixes SSL test setup by skipping early when stunnel is unavailable and correcting the OpenSSL test to use the truststore matching the target SSL endpoint.

Stabilizes several command tests by removing a brittle cluster pub/sub assertion, adding HOTKEYS sampling to guarantee capture, making absolute-TTL assertions tolerant to clock skew, broadening expected failures in concurrent multi-DB tests, and using approximate comparisons for vector similarity scores.

Reviewed by Cursor Bugbot for commit cbeb5f0. Bugbot is set up for automated code reviews on this repo. Configure here.

viktoriya-kutsarova and others added 2 commits April 16, 2026 16:26
…match, TEST_WORK_FOLDER propagation, and floating-point assertions
@tishun tishun force-pushed the fix/flaky-integration-tests branch from 1c315fc to da103f5 Compare April 16, 2026 13:26
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Reviewed by Cursor Bugbot for commit 935053f. Configure here.

@tishun tishun merged commit ff6eec4 into redis:main Apr 16, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants