refactor: task handlers

[declark1]

Major changes:

• Implements new task handlers using reusable common tasks where possible
  • Reduces code duplication and lines of code for each implementation
  • Better organization, one file per handler
  • Simplified error handling, propagating Result<T, Error> through tasks
  • Added placeholder for chat completions streaming task handler (not yet implemented)
• Drop all existing task handlers
• Move task types to handler files
• Update route handlers to call new task handlers

Other Changes:

• Fix default thresholds for common detector tasks
• Add apply_chunk_offset parameter and fix offsets for detect_text_contents task
• Set detector_id in client helpers instead of tasks
• Drop input_id from common detector tasks except text_contents_detections (not needed)
• Drop Option from DetectorConfig fields to improve ergonomics
• Add missing Detection / Detections conversions
• Use buffer size 128 for mpsc and broadcast channels (reasonable default)
• Add current_timestamp util function
• Add Task struct (not yet used, will adopt in next PR as there are cleanups needed first for request types)
• Update chat completions detections to return None instead of Some(ChatDetections { ..default }) when there are no detections, update test accordingly

[evaline-ju]

🎉 Nice, what a net negative diff! Couple of small comments and generally want to confirm:
(1) functional tests have been run against this change and still pass
(2) telemetry traces are expected [as noted separately]

[declark1]

🎉 Nice, what a net negative diff! Couple of small comments and generally want to confirm: (1) functional tests have been run against this change and still pass (2) telemetry traces are expected [as noted separately]

Thanks @evaline-ju. I'm working on addressing your comments inline and diving into the tracing issue, which will require some changes to fix. We do still need to run the functional tests, so it would be good if someone can help with this in the meantime while I work on the tracing, or it can wait until after that is resolved if you prefer.

[declark1]

I made some additional changes for tracing and this is now ready for final review. The primary issue was that span context was not being injected into spawned sub-tasks. A bug fix was also implemented to ensure proper handling of no detectors scenarios in (streaming)_classification_with_gen.

While testing tracing further locally with jaeger, I noticed that we were creating an excessive number of spans as essentially every function in the call stack was instrumented and creating a new span, rather than spans corresponding to a "unit of work". We were also instrumenting some functions that didn't need to be.

I did some limited cleanup to:

1. Reduce the number of spans being created for each request
2. Avoid logging essentially the same or redundant information at multiple layers
3. Changed the default logging format from "full" to "compact"
4. Reduced "incoming_orchestrator_http_request" being logged for every line from the parent span to "request"

There are opportunities for further enhancements, but we can address them later as this is out of the scope of this PR.

[evaline-ju]

LGTM - thanks for the updates!