Deluge of errors when making concurrent translate requests

[clemens1483]

We have been using the gem on version 2.5.3 without any issues. However, we are now trying to update to the newest version, 3.0.2 and are facing quite a few issues

When making many ~10+ simultaneous translation requests via DeepL.translate from within sidekiq workers running in parallel
a lot of the requests fail and raise an error with either

• IOError: stream closed in another thread
• FrozenError: can't modify frozen OpenSSL::SSL::SSLContext

This was NOT an issue on 2.5.3

VERSION: 3.0.2
API CALL: `DeepL.translate("some string", nil, "FR", tag_handling: 'html')

2024-10-10T14:50:19.914Z pid=19199 tid=29nz WARN: IOError: stream closed in another thread
2024-10-10T14:50:19.914Z pid=19199 tid=29nz WARN: /gems/ruby/3.1.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:229:in `wait_readable'
/gems/ruby/3.1.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:229:in `rbuf_fill'
/gems/ruby/3.1.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:199:in `readuntil'
/gems/ruby/3.1.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:209:in `readline'
/usr/local/lib/ruby/3.1.0/net/http/response.rb:42:in `read_status_line'
/usr/local/lib/ruby/3.1.0/net/http/response.rb:31:in `read_new'
/usr/local/lib/ruby/3.1.0/net/http.rb:1609:in `block in transport_request'
/usr/local/lib/ruby/3.1.0/net/http.rb:1600:in `catch'
/usr/local/lib/ruby/3.1.0/net/http.rb:1600:in `transport_request'
/usr/local/lib/ruby/3.1.0/net/http.rb:1573:in `request'
/usr/local/lib/ruby/3.1.0/net/http.rb:1566:in `block in request'
/usr/local/lib/ruby/3.1.0/net/http.rb:985:in `start'
/usr/local/lib/ruby/3.1.0/net/http.rb:1564:in `request'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/base.rb:54:in `block in execute_request_with_retries'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/base.rb:53:in `loop'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/base.rb:53:in `execute_request_with_retries'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/translate.rb:44:in `request'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl.rb:77:in `translate'

2024-10-10T14:50:28.408Z pid=19199 tid=28kj WARN: FrozenError: can't modify frozen OpenSSL::SSL::SSLContext: #<OpenSSL::SSL::SSLContext:0x0000ffff848e46b0 @verify_mode=1, @verify_hostname=true, @max_proto_version=nil, @min_proto_version=769, @cert_store=#<OpenSSL::X509::Store:0x0000ffff8cb26ec0 @verify_callback=nil, @error=nil, @error_string=nil, @chain=nil, @time=nil>, @session_new_cb=#<Proc:0x0000ffff848e4318 /usr/local/lib/ruby/3.1.0/net/http.rb:1061>>
2024-10-10T14:50:28.408Z pid=19199 tid=28kj WARN: /gems/ruby/3.1.0/gems/openssl-3.2.0/lib/openssl/ssl.rb:148:in `options='
/gems/ruby/3.1.0/gems/openssl-3.2.0/lib/openssl/ssl.rb:148:in `set_params'
/usr/local/lib/ruby/3.1.0/net/http.rb:1054:in `connect'
/usr/local/lib/ruby/3.1.0/net/http.rb:995:in `do_start'
/usr/local/lib/ruby/3.1.0/net/http.rb:984:in `start'
/usr/local/lib/ruby/3.1.0/net/http.rb:1564:in `request'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/base.rb:54:in `block in execute_request_with_retries'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/base.rb:53:in `loop'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/base.rb:53:in `execute_request_with_retries'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl/requests/translate.rb:44:in `request'
/gems/ruby/3.1.0/gems/deepl-rb-3.0.2/lib/deepl.rb:77:in `translate'

[JanEbbing]

Thanks for the report! There is some discussion on this in #3 . I don't think the keepalive support I added in 3.0 is possible without such errors with Net::HTTP which we currently use, so if you need to make concurrent requests I'm afraid you will need to keep using 2.5.3 for now - I will look into a HTTP library we can use that supports both.

[foobear]

We also encountered these when using Threads to translate a list of texts with different contexts each.
We will be downgrading to 2.5.3 for now.

If you're looking for a recommendation: I'm quite happy with the http gem. I found it to be more robust than Net::HTTP with a very usable API. It also has built-in keepalive support: https://github.com/httprb/http/wiki/Persistent-Connections-(keep-alive)

[foobear]

We had to go back to 3.2.0 just for the debounce/retry logic, because 2.5.3 keeps running into HTTP 429 when translating many texts.

Background: Occasionally, we are translating about 100 texts, but in chunks of ~5 texts which each have their own translation context, so we cannot batch all 100 texts together into a single request. We saw much better performance sending requests in parallel using threads, instead of sending them on a single keepalive session. We can't prepare those transactions up front, since they happen on user interaction, which is why we care about performance.

So, at least for the time being, we are patching the gem's http_client method to be a single instance per thread.
This resolves the can't modify frozen OpenSSL::SSL::SSLContext issues for us.
It might also avoid stream closed in another thread, but I never encountered those.

DeepL::API.prepend(Module.new do

  def http_client
    Thread.current[:deepl_api_http_client_copy] ||= @http_client.dup
  end

end)

I'm aware that @http_client.dup slows down single DeepL.translate calls, but only slightly.

If anyone else wants to use the monkey-patch above, I suggest also checking the gem version so can verify if the patch is still required for newer releases. Example:

if Gem.loaded_specs['deepl-rb'].version > Gem::Version.new('3.2.0')
  raise ...
else
  DeepL::API.prepend(...)
end

[tim-wovn]

At my work, last year we tried to upgrade to 3.x but had to downgrade for the same reason as reported here. We're staying on 2.x for the time being.

[SirBernardPhilip]

Hey! Just wondering if there are any fixes in progress or we should resort to the monkey-patch posted above?

[chriso0710]

Hi all, any news on this?

I am running into the same problems with Rails 8 and solidqueue, which translates in parallel running threads. Do you recommend downgrading to 2.5.3 or is there a better option?

At the moment we are running our jobs with the following queue config in rails:

- queues: "sequential_translate"
  threads: 1
  processes: 1
  polling_interval: 0.1

which avoids the error, although it forces the jobs into sequential order.

[JanEbbing]

If you need concurrent requests, you'll need to downgrade to 2.5.3 atm, sorry.

I'll try to fit in improvements to CLs into our coming sprints, we're just mostly working on API features at the moment, and our other (larger) libraries don't even have any async support at the moment.

[chriso0710]

Thanks @JanEbbing
For our project, it would be more important to have concurrent requests, rather than getting new features. The gem has all the features we need right now, and some that we don't ;-)

[foobear]

From a technical standpoint, you should be fine using version 3.2 and caching the HTTP client instance once per thread, instead of globally by using the monkey patch from #6 (comment).

The increased memory footprint should be barely noticable and it allows for using v3 features.
For what it's worth: we're using this approach in production on an application with significant user and thread count, without any issues.

(Still, I'd prefer an official solution to a monkey patch. 🙂)