Delay the polling of an SQS message with batchItemFailures feature

I’m using an LambdaToSqsToLambda pattern

Now, the second Lambda (the one that polls messages from SQS), can fail, and I’m using the BatchItemFailures to report the messages that failed to SQS, and letting SQS re-insert those messages back in the Queue to be processed again.

Here’s the AWS CDK code:

    new LambdaToSqsToLambda(stack, `id1`, {
        existingProducerLambdaObj: lambdaA,
        existingConsumerLambdaObj: lambdaB,
        queueProps: {
            queueName: `Queue`,
            retentionPeriod: Duration.days(1),
            visibilityTimeout: Duration.hours(3),
            receiveMessageWaitTime: Duration.seconds(20),
            deliveryDelay: Duration.minutes(10),
        },
        sqsEventSourceProps: {
            maxConcurrency: 2,
            batchSize: 1,
            maxBatchingWindow: Duration.minutes(5),
            reportBatchItemFailures: true,
        },

The lambdaA will place some SQS messages in the Queue, and that’s fine.

The lambdaB, will then poll those messages from SQS, and “slowly” process them.

    async handler(event: SQSEvent) {
        const allRecords = this.sqs.toMessagesBodies(event)
        const batchItemFailures: SQSBatchItemFailure[] = [];

        for (let record of allRecords) {
            try {
                await this.process(record.body);
            } catch (e) {
                console.error('Error: ' + e);
                batchItemFailures.push({
                    itemIdentifier: record.recordId,
                });
            }
        }

        return {
            batchItemFailures: removeEmptyFromArray(batchItemFailures)
        }
    }

My specific need is to process the messages from SQS very slowly, ideally taking up to 1 hour to process the last message out of over 60 inserted messages. This is mainly to address RATE_LIMIT issues.

The problem I’m encountering is that SQS re-inserts the failed messages into the queue, but Lambda polls them immediately after, causing continuous and immediate processing. I’d like to implement a backoff retry approach, even if it’s a static delay (e.g., every 5 minutes poll this message from SQS).

Any help or suggestions would be greatly appreciated.

Thank you!

I’ve tried playing around with queueProps (visibilityTimeout, receiveMessageWaitTime, deliveryDelay and retentionPeriod), but nothing really seem helping.

Also tried changing increasing the maxBatchingWindow of sqsEventSourceProps, but could not see significant results.