|
4 | 4 | :project-owner: redis-field-engineering
|
5 | 5 | :project-name: redis-flink-connector
|
6 | 6 | :project-group: com.redis
|
7 |
| -:project-version: 0.0.7 |
| 7 | +:project-version: 0.0.10 |
8 | 8 | :dist-repo-name: redis-flink-connector-dist
|
9 | 9 |
|
10 | 10 | The Redis Flink Connector is a highly performant, scalable Flink Source and Sink
|
@@ -101,9 +101,12 @@ The following table describes the fields in that class:
|
101 | 101 | | `topicName` | `String` | N/A | Yes | the Topic Name
|
102 | 102 | | `numPartitions` | `int` | N/A | Yes | the number of partitions
|
103 | 103 | | `flushOnCheckpoint` | `boolean` | `false` | No | whether to flush writes on checkpoint
|
| 104 | +| `numMessagesToBuffer` | `int` | `0` | No | number of messages to buffer before flushing (when > 0) |
| 105 | +| `flushIntervalMillis` | `int` | `0` | No | time interval in milliseconds for periodic flushing (when > 0) |
104 | 106 | | `failedSerializationStreamName` | `String` | `""` (empty string) | No | the stream name to serialization errors to
|
105 | 107 | |`useTls` | `boolean` | `false` | No | whether to use TLS
|
106 | 108 | |`mtlsParameters` | `MtlsParameters` | `null` | No | parameters to use for mTLS
|
| 109 | +|`traceConfiguration` | `SinkTraceConfiguration` | `null` | No | configuration for tracing events processed by the sink |
107 | 110 | |===
|
108 | 111 |
|
109 | 112 | You then have to initialize the builder and sink to it:
|
@@ -170,6 +173,49 @@ docker compose up -d
|
170 | 173 |
|
171 | 174 | This will spin up Redis, a Flink Job Manager and Task Manager, and start a Job with Redis as the Source and Sink.
|
172 | 175 |
|
| 176 | +== Sink Trace Configuration |
| 177 | + |
| 178 | +The Redis Sink supports configurable tracing of events processed by the sink. This allows you to track the success or failure of event writes with different levels of detail. Tracing can be configured using the `SinkTraceConfiguration` class with the following options: |
| 179 | + |
| 180 | +=== Trace Levels |
| 181 | + |
| 182 | +The `SinkTraceLevel` enum controls which events are traced: |
| 183 | + |
| 184 | +[cols="1,2",options="header"] |
| 185 | +|=== |
| 186 | +| **Level** | **Description** |
| 187 | +| `NONE` | No tracing (default when configuration is null) |
| 188 | +| `FAILURES` | Only trace failed writes |
| 189 | +| `ALL` | Trace all events (both successful and failed writes) |
| 190 | +|=== |
| 191 | + |
| 192 | +=== Trace Contents |
| 193 | + |
| 194 | +The `SinkTraceContents` enum controls what information is included in the trace: |
| 195 | + |
| 196 | +[cols="1,2",options="header"] |
| 197 | +|=== |
| 198 | +| **Contents** | **Description** |
| 199 | +| `NONE` | No content (minimal tracing) |
| 200 | +| `METADATA` | Include metadata about the event (timestamps, keys, etc.) |
| 201 | +| `ALL` | Include full message contents along with metadata |
| 202 | +|=== |
| 203 | + |
| 204 | +=== Example Configuration |
| 205 | + |
| 206 | +[source,java] |
| 207 | +---- |
| 208 | +// Configure tracing for all events with full message contents |
| 209 | +SinkTraceConfiguration traceConfig = new SinkTraceConfiguration(SinkTraceLevel.ALL, SinkTraceContents.ALL); |
| 210 | +
|
| 211 | +// Use the configuration when building your sink |
| 212 | +RedisSinkConfig config = RedisSinkConfig.builder() |
| 213 | + .host("redis") |
| 214 | + .topicName("output") |
| 215 | + .numPartitions(3) |
| 216 | + .traceConfiguration(traceConfig) |
| 217 | + .build(); |
| 218 | +---- |
173 | 219 |
|
174 | 220 | == Support
|
175 | 221 |
|
|
0 commit comments