Skip to content

add: Doc for Smart Retry Middleware #460

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions docs/available-components/middlewares.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,57 @@ async def test():

`retry_on_error` enables retries for a task. `max_retries` is the maximum number of times,.

## Smart Retry Middleware

The `SmartRetryMiddleware` automatically retries tasks with flexible delay settings and retry strategies when errors occur. This is particularly useful when tasks fail due to temporary issues, such as network errors or temporary unavailability of external services.

### Key Features:

* **Retry Limits**: Set the maximum number of retry attempts (`max_retries`).
* **Delay Before Retry**: Define a fixed delay or use additional strategies.
* **Jitter**: Adds random delay variations to distribute retries evenly and prevent simultaneous task execution.
* **Exponential Backoff**: Increases the delay for each subsequent retry, useful for gradually recovering system performance.
* **Custom Schedule Source (`schedule_source`)**: Enables using a custom schedule source to manage delayed task execution.

### Middleware Integration

To use `SmartRetryMiddleware`, add it to the list of middlewares in your broker:

```python
from taskiq import ZeroMQBroker
from taskiq.middlewares import SmartRetryMiddleware

broker = ZeroMQBroker().with_middlewares(
SmartRetryMiddleware(
default_retry_count=5,
default_delay=10,
use_jitter=True,
use_delay_exponent=True,
max_delay_exponent=120
),
)
```

### Using Middleware with Tasks

To enable retries for a specific task, specify the following parameters:

```python
@broker.task(retry_on_error=True, max_retries=10, delay=15)
async def my_task():
raise Exception("Error, retrying!")
```

* `retry_on_error`: Enables the retry mechanism for the specific task.
* `max_retries`: Maximum number of retries (overrides middleware default).
* `delay`: Initial delay before retrying the task, in seconds.

### Usage Recommendations

Use jitter and exponential backoff to avoid repetitive load peaks, especially in high-load systems. Choose appropriate `max_delay_exponent` values to prevent excessively long intervals between retries if your task execution is time-sensitive.



### Prometheus middleware

You can enable prometheus metrics for workers by adding PrometheusMiddleware.
Expand Down