Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix connection pool bug which can cause delays in obtaining a connection #3079

Merged
merged 2 commits into from
Apr 9, 2025

Conversation

andreachild
Copy link
Contributor

@andreachild andreachild commented Apr 1, 2025

It has been observed in some environments that integration tests such as GraphBinaryGroovyRemoteFeatureTest can encounter 16 second delays between test executions. Investigation shows this can occur if one thread is attempting to obtain a connection and another thread is creating or releasing a connection at the same time. It is possible for the thread releasing or creating a connection to signal the hasAvailableConnection condition after the other thread has determined there are no available connections but before calling await on the condition. Thus the thread obtaining the connection will wait for default timeout of 16 seconds before attempting to retrieve a connection again, causing the observed delays.

This fix involves locking the waitLock before obtaining and potentially calling await for an available connection which prevents other threads from signalling the hasAvailableConnection before other threads call await. The fix was tested by observing integration test execution on environments which previously produced the 16 second delays and also by executing ProfilingApplication. The results of ProfilingApplication did not demonstrate any degradation of average requests per second.

@codecov-commenter
Copy link

codecov-commenter commented Apr 1, 2025

Codecov Report

Attention: Patch coverage is 66.66667% with 5 lines in your changes missing coverage. Please review.

Project coverage is 78.11%. Comparing base (cfd6889) to head (65e6f1f).
Report is 200 commits behind head on master.

Files with missing lines Patch % Lines
...pache/tinkerpop/gremlin/driver/ConnectionPool.java 71.42% 2 Missing and 2 partials ⚠️
...rpop/gremlin/driver/util/ProfilingApplication.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3079      +/-   ##
============================================
+ Coverage     77.87%   78.11%   +0.24%     
- Complexity    13578    13969     +391     
============================================
  Files          1015     1019       +4     
  Lines         59308    60036     +728     
  Branches       6835     6971     +136     
============================================
+ Hits          46184    46896     +712     
+ Misses        10817    10816       -1     
- Partials       2307     2324      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

… a thread attempting to retrieve a connection calls await, causing 16 second wait for obtaining a connection. This scenario is now prevented by locking before obtaining and potentially calling await for an available connection.
@andreachild andreachild changed the title Draft: fix for connection pool bug where the waitLock can be signaled … Fix connection pool bug which can cause delays in obtaining a connection Apr 4, 2025
@andreachild andreachild marked this pull request as ready for review April 4, 2025 21:16
@Cole-Greer
Copy link
Contributor

VOTE +1

2 similar comments
@kenhuuu
Copy link
Contributor

kenhuuu commented Apr 9, 2025

VOTE +1

@xiazcy
Copy link
Contributor

xiazcy commented Apr 9, 2025

VOTE +1

@Cole-Greer Cole-Greer merged commit 30dca96 into apache:master Apr 9, 2025
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants