Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: retry on retryable httpx-related error to improve evaluation stability #7719

Merged
merged 15 commits into from
Apr 6, 2025

Conversation

xingyaoww
Copy link
Collaborator

@xingyaoww xingyaoww commented Apr 5, 2025

  • This change is worth documenting at https://docs.all-hands.dev/
  • Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

End-user friendly description of the problem this fixes or functionality that this introduces.


Give a summary of what the PR does, explaining any non-trivial design decisions.

This PR makes sure we retry on retryable httpx related error for all action execution request, rather than just on "/alive" request. This will help improve the stability of our evaluation harness.


Link of any specific issues this addresses.


To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:4301467-nikolaik   --name openhands-app-4301467   docker.all-hands.dev/all-hands-ai/openhands:4301467

@xingyaoww xingyaoww requested review from tofarr and enyst April 5, 2025 20:56
@xingyaoww xingyaoww changed the title fix: retry on retryable httpx-related error fix: retry on retryable httpx-related error to improve evaluation stability Apr 5, 2025
Copy link
Contributor

@neubig neubig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are 3 changes:

@xingyaoww xingyaoww enabled auto-merge (squash) April 5, 2025 22:03
@xingyaoww xingyaoww merged commit 288bcd2 into main Apr 6, 2025
17 checks passed
@xingyaoww xingyaoww deleted the xw/fix-eval-retry branch April 6, 2025 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants