-
Notifications
You must be signed in to change notification settings - Fork 1k
first commit - lambda-durable-functions-nodejs-sam #2935
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,279 @@ | ||||||
| # AWS Lambda Durable Functions with Node.js | ||||||
|
|
||||||
| This pattern demonstrates AWS Lambda Durable Functions using Node.js to build resilient, long-running workflows that can execute for up to one year. | ||||||
|
|
||||||
| Learn more about this pattern at Serverless Land Patterns: [https://serverlessland.com/patterns/lambda-durable-functions-nodejs-sam](https://serverlessland.com/patterns/lambda-durable-functions-nodejs-sam) | ||||||
|
|
||||||
| Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example. | ||||||
|
|
||||||
| ## Requirements | ||||||
|
|
||||||
| * [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources. | ||||||
| * [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured | ||||||
| * [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) | ||||||
| * [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed | ||||||
|
|
||||||
| ## Deployment Instructions | ||||||
|
|
||||||
| 1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository: | ||||||
| ``` | ||||||
| git clone https://github.com/aws-samples/serverless-patterns | ||||||
| ``` | ||||||
| 1. Change directory to the pattern directory: | ||||||
| ``` | ||||||
| cd lambda-durable-functions-nodejs-sam | ||||||
| ``` | ||||||
| 1. From the command line, use AWS SAM to build and deploy the AWS resources for the pattern as specified in the template.yml file: | ||||||
| ``` | ||||||
| sam build | ||||||
| sam deploy --guided | ||||||
| ``` | ||||||
| 1. During the prompts: | ||||||
| * Enter a stack name | ||||||
| * Enter the desired AWS Region (must support Lambda Durable Functions) | ||||||
| * Allow SAM CLI to create IAM roles with the required permissions. | ||||||
|
|
||||||
| Once you have run `sam deploy --guided` mode once and saved arguments to a configuration file (samconfig.toml), you can use `sam deploy` in future to use these defaults. | ||||||
|
|
||||||
| 1. Note the outputs from the SAM deployment process. These contain the resource names and/or ARNs which are used for testing. | ||||||
|
|
||||||
| ## How it works | ||||||
|
|
||||||
| This pattern demonstrates AWS Lambda Durable Functions using Node.js. It implements a simple order processing workflow with automatic checkpointing, durable waits, and fault tolerance. | ||||||
|
|
||||||
| The orchestrator function uses the `@aws/durable-execution-sdk-js` to implement: | ||||||
| - Checkpointed steps with `context.step()` | ||||||
| - Durable waits with `context.wait()` | ||||||
| - Automatic recovery from failures | ||||||
| - Structured JSON logging | ||||||
|
|
||||||
| The workflow: | ||||||
| 1. Validates input | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why isn't the validation checkpointed? |
||||||
| 2. Executes enrichment step (checkpointed) by invoking the OrderEnricher Lambda | ||||||
| 3. Waits 2 seconds (durable wait - no compute charges) | ||||||
| 4. Executes finalization step (checkpointed) | ||||||
| 5. Returns result | ||||||
|
|
||||||
| ## Architecture | ||||||
|
|
||||||
| This demo includes two Lambda functions: | ||||||
|
|
||||||
| 1. **DurableOrderProcessor** (Orchestrator) | ||||||
| - Uses `@aws/durable-execution-sdk-js` for durable execution | ||||||
| - Implements checkpointed steps with `context.step()` | ||||||
| - Demonstrates durable wait with `context.wait()` | ||||||
| - Includes structured JSON logging | ||||||
| - Comprehensive error handling | ||||||
|
Comment on lines
+62
to
+66
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Didn't you just write this in lines 44-48? Seems redundant |
||||||
|
|
||||||
| 2. **OrderEnricher** (Worker) | ||||||
| - Simple Lambda function (non-durable) | ||||||
| - Enriches order data with customer information | ||||||
| - Called by the orchestrator | ||||||
|
|
||||||
| ## Testing | ||||||
|
|
||||||
| After deployment, test the durable function: | ||||||
|
|
||||||
| ### 1. Get Function ARNs from Stack Outputs | ||||||
|
|
||||||
| ```bash | ||||||
| # Get the orchestrator ARN (with :prod alias) | ||||||
| aws cloudformation describe-stacks \ | ||||||
| --stack-name STACK_NAME \ | ||||||
| --query 'Stacks[0].Outputs[?OutputKey==`DurableOrderProcessorArn`].OutputValue' \ | ||||||
| --output text | ||||||
|
|
||||||
| # Get the enricher ARN | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| aws cloudformation describe-stacks \ | ||||||
| --stack-name STACK_NAME \ | ||||||
| --query 'Stacks[0].Outputs[?OutputKey==`OrderEnricherArn`].OutputValue' \ | ||||||
| --output text | ||||||
| ``` | ||||||
|
|
||||||
| ### 2. Invoke the Durable Function | ||||||
|
|
||||||
| ```bash | ||||||
| # Create test payload | ||||||
| cat > test-payload.json << EOF | ||||||
| { | ||||||
| "orderId": "ORDER-123", | ||||||
| "nodejsLambdaArn": "arn:aws:lambda:REGION:ACCOUNT_ID:function:STACK_NAME-OrderEnricher" | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Doesn't the user need to replace these values?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And why do it at all if you automatically generate the test command as stack output? |
||||||
| } | ||||||
| EOF | ||||||
|
|
||||||
| # Invoke (replace with your actual ARN from step 1) | ||||||
| aws lambda invoke \ | ||||||
| --function-name arn:aws:lambda:REGION:ACCOUNT_ID:function:STACK_NAME-DurableOrderProcessor:prod \ | ||||||
| --payload file://test-payload.json \ | ||||||
| --cli-binary-format raw-in-base64-out \ | ||||||
| response.json | ||||||
|
|
||||||
| # View response | ||||||
| cat response.json | ||||||
| ``` | ||||||
|
|
||||||
| ### 3. View Logs | ||||||
|
|
||||||
| ```bash | ||||||
| # View orchestrator logs | ||||||
| aws logs tail /aws/lambda/STACK_NAME-DurableOrderProcessor --follow | ||||||
|
|
||||||
| # View enricher logs | ||||||
| aws logs tail /aws/lambda/STACK_NAME-OrderEnricher --follow | ||||||
| ``` | ||||||
|
|
||||||
| ## Expected Output | ||||||
|
|
||||||
| Successful execution returns: | ||||||
|
|
||||||
| ```json | ||||||
| { | ||||||
| "success": true, | ||||||
| "orderId": "ORDER-123", | ||||||
| "enrichmentResult": { | ||||||
| "statusCode": 200, | ||||||
| "orderId": "ORDER-123", | ||||||
| "enrichedData": { | ||||||
| "customerId": "CUST-XXXX", | ||||||
| "timestamp": "2026-02-08T20:58:24.548Z" | ||||||
| } | ||||||
| }, | ||||||
| "finalResult": { | ||||||
| "orderId": "ORDER-123", | ||||||
| "status": "COMPLETED", | ||||||
| "enrichedData": { ... }, | ||||||
| "finalizedAt": "2026-02-08T20:58:26.859Z", | ||||||
| "message": "Order finalized successfully" | ||||||
| }, | ||||||
| "message": "Order processed successfully with durable execution", | ||||||
| "processedAt": "2026-02-08T20:58:26.954Z" | ||||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| ## Observing Durable Execution | ||||||
|
|
||||||
| Check CloudWatch Logs to see the durable execution in action: | ||||||
|
|
||||||
| 1. **First Invocation**: Executes enrichment step, hits wait, suspends | ||||||
| 2. **Second Invocation** (~2 seconds later): Resumes from checkpoint, skips enrichment (uses stored result), completes finalization | ||||||
|
|
||||||
| You'll notice: | ||||||
| - Multiple Lambda invocations for a single workflow | ||||||
| - Enrichment step result is reused (not re-executed) | ||||||
| - Total execution time includes the 2-second wait, but you only pay for active compute time | ||||||
|
|
||||||
| ## Cleanup | ||||||
|
|
||||||
| To remove all resources: | ||||||
|
|
||||||
| ```bash | ||||||
| sam delete --stack-name STACK_NAME | ||||||
| ``` | ||||||
|
|
||||||
| Or via CloudFormation: | ||||||
|
|
||||||
| ```bash | ||||||
| aws cloudformation delete-stack --stack-name STACK_NAME | ||||||
|
Comment on lines
+173
to
+176
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. unnecessary |
||||||
| ``` | ||||||
|
|
||||||
| ## Additional Information | ||||||
|
|
||||||
| ### Key Features Demonstrated | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This seems to add little value as you are just repeating the code. Consider adding comments to the code directly (if at all) |
||||||
|
|
||||||
| 1. **Checkpointed Steps** | ||||||
| ```javascript | ||||||
| const enrichmentResult = await context.step('enrich-order', async () => { | ||||||
| return await invokeNodejsLambda(nodejsLambdaArn, orderId, logger); | ||||||
| }); | ||||||
| ``` | ||||||
|
|
||||||
| 2. **Durable Wait** | ||||||
| ```javascript | ||||||
| await context.wait({ seconds: 2 }); | ||||||
| ``` | ||||||
|
|
||||||
| 3. **Structured Logging** | ||||||
| ```javascript | ||||||
| logger.info('Starting durable order processing', { | ||||||
| event, | ||||||
| remainingTimeMs: context.getRemainingTimeInMillis?.() | ||||||
| }); | ||||||
| ``` | ||||||
|
|
||||||
| 4. **Error Handling** | ||||||
| ```javascript | ||||||
| try { | ||||||
| // Workflow logic | ||||||
| } catch (error) { | ||||||
| logger.error('Order processing failed', { | ||||||
| error: error.message, | ||||||
| errorName: error.name | ||||||
| }); | ||||||
| return { success: false, error: { ... } }; | ||||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| ### Customization Options | ||||||
|
|
||||||
| **Modify Wait Duration:** | ||||||
| ```javascript | ||||||
| // Wait for 5 minutes | ||||||
| await context.wait({ minutes: 5 }); | ||||||
|
|
||||||
| // Wait for 1 hour | ||||||
| await context.wait({ hours: 1 }); | ||||||
|
|
||||||
| // Wait for 2 days | ||||||
| await context.wait({ days: 2 }); | ||||||
| ``` | ||||||
|
|
||||||
| **Add More Steps:** | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Unnecessary, remove |
||||||
| ```javascript | ||||||
| const step1Result = await context.step('step-1', async () => { | ||||||
| // Your logic here | ||||||
| }); | ||||||
|
|
||||||
| const step2Result = await context.step('step-2', async () => { | ||||||
| // Your logic here | ||||||
| }); | ||||||
| ``` | ||||||
|
|
||||||
| **Configure Retry Behavior:** | ||||||
| ```javascript | ||||||
| const result = await context.step('my-step', async () => { | ||||||
| // Your logic | ||||||
| }, { | ||||||
| maxAttempts: 3, | ||||||
| backoffRate: 2.0, | ||||||
| intervalSeconds: 1 | ||||||
| }); | ||||||
| ``` | ||||||
|
|
||||||
| ### Supported Regions | ||||||
|
|
||||||
| AWS Lambda Durable Functions are available in select regions. Check the [official documentation](https://docs.aws.amazon.com/lambda/latest/dg/durable-supported-runtimes.html) for the latest list. | ||||||
|
|
||||||
| As of February 2026, supported regions include: | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why even add the incomplete list if you could add a link to the docs which will be kept up-to-date. However, the above link lists runtimes not regions. Add a link to the Capabilities by Region page |
||||||
| - us-east-1 (N. Virginia) | ||||||
| - us-west-2 (Oregon) | ||||||
| - eu-west-1 (Ireland) | ||||||
| - ap-southeast-1 (Singapore) | ||||||
|
|
||||||
| ### Troubleshooting | ||||||
|
|
||||||
| **"InvalidParameterValueException: You cannot invoke a durable function using an unqualified ARN"** | ||||||
| - Solution: Always use a qualified ARN (with version or alias). This template automatically creates a `prod` alias. | ||||||
|
|
||||||
| **"Cannot find module '@aws/durable-execution-sdk-js'"** | ||||||
| - Solution: Ensure dependencies are installed. SAM CLI automatically runs `npm install` during build. | ||||||
|
|
||||||
| **Function times out** | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why would that happen? |
||||||
| - Solution: Increase the timeout in `template.yaml`: | ||||||
| ```yaml | ||||||
| Timeout: 900 # 15 minutes | ||||||
| ``` | ||||||
|
|
||||||
| ---- | ||||||
| Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
|
|
||||||
| SPDX-License-Identifier: MIT-0 | ||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,63 @@ | ||||||
| { | ||||||
| "title": "AWS Lambda Durable Functions with Node.js", | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please make sure to always use the correct capitalization |
||||||
| "description": "Demonstrates AWS Lambda Durable Functions using Node.js with automatic checkpointing, durable waits, and fault tolerance for long-running workflows.", | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| "language": "Node.js", | ||||||
| "level": "200", | ||||||
| "framework": "SAM", | ||||||
| "introBox": { | ||||||
| "headline": "How it works", | ||||||
| "text": [ | ||||||
| "This pattern demonstrates AWS Lambda Durable Functions using Node.js to build resilient, long-running workflows that can execute for up to one year.", | ||||||
| "The orchestrator function uses the @aws/durable-execution-sdk-js to implement checkpointed steps with context.step(), durable waits with context.wait(), and automatic recovery from failures.", | ||||||
| "The workflow validates input, invokes an enrichment Lambda function, waits 2 seconds (without compute charges), and finalizes the order processing." | ||||||
| ] | ||||||
| }, | ||||||
| "gitHub": { | ||||||
| "template": { | ||||||
| "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/lambda-durable-functions-nodejs-sam", | ||||||
| "templateURL": "serverless-patterns/lambda-durable-functions-nodejs-sam", | ||||||
| "projectFolder": "lambda-durable-functions-nodejs-sam", | ||||||
| "templateFile": "template.yaml" | ||||||
| } | ||||||
| }, | ||||||
| "resources": { | ||||||
| "bullets": [ | ||||||
| { | ||||||
| "text": "AWS Lambda Durable Functions Documentation", | ||||||
| "link": "https://docs.aws.amazon.com/lambda/latest/dg/durable-functions.html" | ||||||
| }, | ||||||
| { | ||||||
| "text": "Durable Execution SDK for JavaScript", | ||||||
| "link": "https://github.com/aws/aws-durable-execution-sdk-js" | ||||||
| }, | ||||||
| { | ||||||
| "text": "AWS SAM Documentation", | ||||||
| "link": "https://docs.aws.amazon.com/serverless-application-model/" | ||||||
| } | ||||||
| ] | ||||||
| }, | ||||||
| "deploy": { | ||||||
| "text": [ | ||||||
| "sam build", | ||||||
| "sam deploy --guided" | ||||||
| ] | ||||||
| }, | ||||||
| "testing": { | ||||||
| "text": [ | ||||||
| "See the GitHub repo for detailed testing instructions." | ||||||
| ] | ||||||
| }, | ||||||
| "cleanup": { | ||||||
| "text": [ | ||||||
| "Delete the stack: sam delete --stack-name durable-functions-demo" | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| ] | ||||||
| }, | ||||||
| "authors": [ | ||||||
| { | ||||||
| "name": "Sangeetha S", | ||||||
| "image": "", | ||||||
| "bio": "Technical Account Manager @ AWS", | ||||||
| "linkedin": "sangita-sethumadhavan" | ||||||
| } | ||||||
| ] | ||||||
| } | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| exports.handler = async (event) => { | ||
| console.log('Received order:', event.orderId); | ||
|
|
||
| // Simple enrichment | ||
| return { | ||
| statusCode: 200, | ||
| orderId: event.orderId, | ||
| enrichedData: { | ||
| customerId: 'CUST-' + Math.floor(Math.random() * 10000), | ||
| timestamp: new Date().toISOString() | ||
| } | ||
| }; | ||
| }; |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "name": "order-enrichment-lambda", | ||
| "version": "1.0.0", | ||
| "description": "Simple order enrichment service", | ||
| "main": "index.js", | ||
| "keywords": [ | ||
| "aws", | ||
| "lambda" | ||
| ], | ||
| "author": "", | ||
| "license": "MIT" | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please make sure to always use the correct capitalization