GeezerVison is showing github.com — docs(batch): explain record type discrepancy in failure and success h… · MistaIA/powertools-lambda-python@920d70e
File tree
Expand file treeCollapse file tree
Open diff view settings
examples/batch_processing/src
Expand file treeCollapse file tree
Open diff view settings
Collapse file
Copy file name to clipboardExpand all lines: docs/utilities/batch.md +11-6Lines changed: 11 additions & 6 deletions - Display the source diff
- Display the rich diff
| Original file line number | Diff line number | Diff line change |
|---|
@@ -522,14 +522,19 @@ You might want to bring custom logic to the existing `BatchProcessor` to slightl |
522 | 522 |
|
523 | 523 | For these scenarios, you can subclass `BatchProcessor` and quickly override `success_handler` and `failure_handler` methods: |
524 | 524 |
|
525 | | -* **`success_handler()`** – Keeps track of successful batch records |
526 | | -* **`failure_handler()`** – Keeps track of failed batch records |
| 525 | +* **`success_handler()`** is called for each successfully processed record |
| 526 | +* **`failure_handler()`** is called for each failed record |
527 | 527 |
|
528 | | -???+ example |
529 | | - Let's suppose you'd like to add a metric named `BatchRecordFailures` for each batch record that failed processing |
| 528 | +???+ note |
| 529 | + These functions have a common `record` argument. For backward compatibility reasons, their type is not the same: |
530 | 530 |
|
531 | | -```python hl_lines="8 9 16-19 22 38" title="Extending failure handling mechanism in BatchProcessor" |
532 | | ---8<-- "examples/batch_processing/src/extending_failure.py" |
| 531 | + - `success_handler`: `record` type is `dict[str, Any]`, the raw record data. |
| 532 | + - `failure_handler`: `record` type can be an Event Source Data Class or your [Pydantic model](#pydantic-integration). During Pydantic validation errors, we fall back and serialize `record` to Event Source Data Class to not break the processing pipeline. |
| 533 | + |
| 534 | +Let's suppose you'd like to add metrics to track successes and failures of your batch records. |
| 535 | + |
| 536 | +```python hl_lines="8-10 18-25 28 44" title="Extending failure handling mechanism in BatchProcessor" |
| 537 | +--8<-- "examples/batch_processing/src/extending_processor_handlers.py" |
533 | 538 | ``` |
534 | 539 |
|
535 | 540 | ### Create your own partial processor |
|
Collapse file
Copy file name to clipboardExpand all lines: examples/batch_processing/src/extending_processor_handlers.py +6Lines changed: 6 additions & 0 deletions | Original file line number | Diff line number | Diff line change |
|---|
|
1 | 1 | import json |
| 2 | +from typing import Any |
2 | 3 |
|
3 | 4 | from aws_lambda_powertools import Logger, Metrics, Tracer |
4 | 5 | from aws_lambda_powertools.metrics import MetricUnit |
|
9 | 10 | FailureResponse, |
10 | 11 | process_partial_response, |
11 | 12 | ) |
| 13 | +from aws_lambda_powertools.utilities.batch.base import SuccessResponse |
12 | 14 | from aws_lambda_powertools.utilities.data_classes.sqs_event import SQSRecord |
13 | 15 | from aws_lambda_powertools.utilities.typing import LambdaContext |
14 | 16 |
|
15 | 17 |
|
16 | 18 | class MyProcessor(BatchProcessor): |
| 19 | + def success_handler(self, record: dict[str, Any], result: Any) -> SuccessResponse: |
| 20 | + metrics.add_metric(name="BatchRecordSuccesses", unit=MetricUnit.Count, value=1) |
| 21 | + return super().success_handler(record, result) |
| 22 | + |
17 | 23 | def failure_handler(self, record: SQSRecord, exception: ExceptionInfo) -> FailureResponse: |
18 | 24 | metrics.add_metric(name="BatchRecordFailures", unit=MetricUnit.Count, value=1) |
19 | 25 | return super().failure_handler(record, exception) |
|