fix: bound the retry budget for two idempotency DynamoDB writes#116
Merged
Conversation
The @idempotent layer makes two sequential DynamoDB writes per request (save_inprogress before the handler, save_success after), so a DynamoDB brownout hits the shared retry budget twice. With total_max_attempts=3 (~7.5s per call) the two writes could reach ~15s and exceed the 10s function timeout — a 502 off the OpenAPI contract that feeds the canary rollback alarm. Drop to total_max_attempts=2 (~4s/call, ~8s for both) and correct the budget comment and the test assertion. Also document (no behavior change) that the RUM guest identity pool lets anyone call rum:PutRumEvents directly, and that session_sample_rate is client-side and does not bound that vector — the real guardrail is a Budgets / RUM-volume alarm.
🏗️ CDK infra diff — PR vs
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses a Well-Architected review (Reliability + Cost).
Finding 1 — Reliability (fixed in code)
The
@idempotentlayer makes two sequential DynamoDB writes per request (save_inprogressPutItembefore the handler,save_successUpdateItemafter), and the persistence layer sharesboto_config. Withtotal_max_attempts=3(~7.5s worst case per call), a DynamoDB brownout hits the budget twice → ~15s > the 10s function timeout → a 502 off the OpenAPI contract that feeds the CodeDeploy canary rollback alarm. The prior budget comment only reasoned about one browned-out call.lambda/app.py:total_max_attempts3 → 2 (~4s/call, ~8s for both writes, < 10s)tests/unit/test_handler.py: assertion 3 → 2 + docstringFinding 2 — RUM guest pool (documented, no behavior change)
The vulnerability (public unauthenticated pool → anyone can mint guest creds and call
rum:PutRumEventsdirectly → uncapped ingestion cost) is real, but the review's proposed remediation — loweringsession_sample_rate— does not mitigate it: that knob is client-side (honored by the RUM browser client); a directPutRumEventscaller ignores it. Lowering it only reduces legitimate traffic (already a separate cost TODO). So instead of a mislabeled sample-rate change:frontend_stack.py: comment atsession_sample_rateexplaining it's client-side and doesn't bound the data planeTODO.md: new item naming the abuse vector and the real guardrail (Budgets / RUM-volume alarm) as a fork responsibilityValidation
make prpasses end to end (tests, cdk-nag clean, OpenAPI in sync). No infra shape or IAM change.