markn.ca
Archive 路 路 3 min read

Making Sure Events Don't Drop In The AWS Cloud

Once you've aggregated all of your AWS Security Hub Findings, here's one way to visualize and analyze them.

Making Sure Events Don't Drop In The AWS Cloud

Event-driven architectures can be really powerful.

A critical aspect of these designs is properly handling the events that drive the solution. You want to make sure that your design gracefully handles events that can鈥檛 be processed when they are emitted.

There are a few patterns to do this. This Twitter thread 馃憞 looks at two methods to address this challenge using Amazon EventBridge.

Tweet 1/11 馃憞 Next tweet

one of the most critical aspects of an event-driven system is making sure that events don't fall on the floor there are two general paths to take here: - make sure unhandled events can be replayed - make sure that unhandled events can be resent 馃У #cloud #devops

@marknca tweeted at 09-Feb-2022, 00:40

Tweet 2/11 馃憞 Next tweet 馃憜 Start

for most use cases, replaying or parking events is a more elegant solution * there are use cases where resending unhandled events is the better path, it entirely depends on what the event is describing let's look at a two ways to implement this @awscloud... 馃У #cloud #devops

@marknca tweeted at 09-Feb-2022, 00:40

Tweet 3/11 馃憞 Next tweet 馃憜 Start

@awscloud the first: dead-letter queues in Amazon Eventbridge the docs are up at https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-rule-dlq.html 馃У #cloud #devops

@marknca tweeted at 09-Feb-2022, 00:40

Tweet 4/11 馃憞 Next tweet 馃憜 Start

@awscloud EventBridge dead-letter queues (DLQ) are actually SQS queues under the covers you don't have to worry about that but it's handy to know, because it does open up some possibilities EventBridge DLQ can be used when EventBridge can't fire a trigger for an event 馃У #cloud #devops

@marknca tweeted at 09-Feb-2022, 00:40

Tweet 5/11 馃憞 Next tweet 馃憜 Start

@awscloud by default, EventBridge will try to send an event again for up to 24 hours or 185 times...whichever comes first when that happens, EventBridge can drop the event to a DLQ * you can send them to the DLQ directly on 1st fail too 馃У #cloud #devops

@marknca tweeted at 09-Feb-2022, 00:40

Tweet 6/11 馃憞 Next tweet 馃憜 Start

@awscloud events in the DLQ can then be drained directly from an @awscloud Lambda function or you could use your own logic to handle them ...use Lambda more at https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-rule-dlq.html#eb-dlq-resend 馃У #cloud #devops

@marknca tweeted at 09-Feb-2022, 00:40

Tweet 7/11 馃憞 Next tweet 馃憜 Start

@awscloud so a DLQ will help you park events in order to resend them the 2nd pattern is to use EventBridge's archive feature...yes, it has an archive feature the docs are up at https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-archive.html 馃У #cloud #devops

@marknca tweeted at 09-Feb-2022, 00:40

Tweet 8/11 馃憞 Next tweet 馃憜 Start

@awscloud the archive is another event target. it has a configurable retention period after which events are deleted once events are in the archive, you can use the replay feature to...um...replay them docs at https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-replay-archived-event.html 馃У #cloud #devops

@marknca tweeted at 09-Feb-2022, 00:40

Tweet 9/11 馃憞 Next tweet 馃憜 Start

@awscloud now the DLQ is a lot more flexible and should be your go-to when trying to avoid dropping events ...but the archive is really interesting. the replay feature is good for series of events over a set timeframe. for some use cases, this is the way to go 馃У #cloud #devops

@marknca tweeted at 09-Feb-2022, 00:40

Tweet 10/11 馃憞 Next tweet 馃憜 Start

@awscloud regardless of the approach, most event-driven solutions don't want to lose events making sure that your design can deal with those unhandled events effectively is critical to a reslient design 馃У #cloud #devops

@marknca tweeted at 09-Feb-2022, 00:40

Tweet 11/11 馃憞 Next tweet 馃憜 Start

@awscloud this is also another example of why you should be reading the docs when you using any cloud service honestly, I had no idea that the archive feature existed in EventBridge. glad i stumbled across it today! /馃У #cloud #devops

@marknca tweeted at 09-Feb-2022, 00:40

馃憜 Start

Share Tweet Share Share Email

Read next