diff --git a/lambda-durable-functions-nodejs-sam/README.md b/lambda-durable-functions-nodejs-sam/README.md new file mode 100644 index 000000000..b7918c2e5 --- /dev/null +++ b/lambda-durable-functions-nodejs-sam/README.md @@ -0,0 +1,279 @@ +# AWS Lambda Durable Functions with Node.js + +This pattern demonstrates AWS Lambda Durable Functions using Node.js to build resilient, long-running workflows that can execute for up to one year. + +Learn more about this pattern at Serverless Land Patterns: [https://serverlessland.com/patterns/lambda-durable-functions-nodejs-sam](https://serverlessland.com/patterns/lambda-durable-functions-nodejs-sam) + +Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example. + +## Requirements + +* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources. +* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured +* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) +* [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed + +## Deployment Instructions + +1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository: + ``` + git clone https://github.com/aws-samples/serverless-patterns + ``` +1. Change directory to the pattern directory: + ``` + cd lambda-durable-functions-nodejs-sam + ``` +1. From the command line, use AWS SAM to build and deploy the AWS resources for the pattern as specified in the template.yml file: + ``` + sam build + sam deploy --guided + ``` +1. During the prompts: + * Enter a stack name + * Enter the desired AWS Region (must support Lambda Durable Functions) + * Allow SAM CLI to create IAM roles with the required permissions. + + Once you have run `sam deploy --guided` mode once and saved arguments to a configuration file (samconfig.toml), you can use `sam deploy` in future to use these defaults. + +1. Note the outputs from the SAM deployment process. These contain the resource names and/or ARNs which are used for testing. + +## How it works + +This pattern demonstrates AWS Lambda Durable Functions using Node.js. It implements a simple order processing workflow with automatic checkpointing, durable waits, and fault tolerance. + +The orchestrator function uses the `@aws/durable-execution-sdk-js` to implement: +- Checkpointed steps with `context.step()` +- Durable waits with `context.wait()` +- Automatic recovery from failures +- Structured JSON logging + +The workflow: +1. Validates input +2. Executes enrichment step (checkpointed) by invoking the OrderEnricher Lambda +3. Waits 2 seconds (durable wait - no compute charges) +4. Executes finalization step (checkpointed) +5. Returns result + +## Architecture + +This demo includes two Lambda functions: + +1. **DurableOrderProcessor** (Orchestrator) + - Uses `@aws/durable-execution-sdk-js` for durable execution + - Implements checkpointed steps with `context.step()` + - Demonstrates durable wait with `context.wait()` + - Includes structured JSON logging + - Comprehensive error handling + +2. **OrderEnricher** (Worker) + - Simple Lambda function (non-durable) + - Enriches order data with customer information + - Called by the orchestrator + +## Testing + +After deployment, test the durable function: + +### 1. Get Function ARNs from Stack Outputs + +```bash +# Get the orchestrator ARN (with :prod alias) +aws cloudformation describe-stacks \ + --stack-name STACK_NAME \ + --query 'Stacks[0].Outputs[?OutputKey==`DurableOrderProcessorArn`].OutputValue' \ + --output text + +# Get the enricher ARN +aws cloudformation describe-stacks \ + --stack-name STACK_NAME \ + --query 'Stacks[0].Outputs[?OutputKey==`OrderEnricherArn`].OutputValue' \ + --output text +``` + +### 2. Invoke the Durable Function + +```bash +# Create test payload +cat > test-payload.json << EOF +{ + "orderId": "ORDER-123", + "nodejsLambdaArn": "arn:aws:lambda:REGION:ACCOUNT_ID:function:STACK_NAME-OrderEnricher" +} +EOF + +# Invoke (replace with your actual ARN from step 1) +aws lambda invoke \ + --function-name arn:aws:lambda:REGION:ACCOUNT_ID:function:STACK_NAME-DurableOrderProcessor:prod \ + --payload file://test-payload.json \ + --cli-binary-format raw-in-base64-out \ + response.json + +# View response +cat response.json +``` + +### 3. View Logs + +```bash +# View orchestrator logs +aws logs tail /aws/lambda/STACK_NAME-DurableOrderProcessor --follow + +# View enricher logs +aws logs tail /aws/lambda/STACK_NAME-OrderEnricher --follow +``` + +## Expected Output + +Successful execution returns: + +```json +{ + "success": true, + "orderId": "ORDER-123", + "enrichmentResult": { + "statusCode": 200, + "orderId": "ORDER-123", + "enrichedData": { + "customerId": "CUST-XXXX", + "timestamp": "2026-02-08T20:58:24.548Z" + } + }, + "finalResult": { + "orderId": "ORDER-123", + "status": "COMPLETED", + "enrichedData": { ... }, + "finalizedAt": "2026-02-08T20:58:26.859Z", + "message": "Order finalized successfully" + }, + "message": "Order processed successfully with durable execution", + "processedAt": "2026-02-08T20:58:26.954Z" +} +``` + +## Observing Durable Execution + +Check CloudWatch Logs to see the durable execution in action: + +1. **First Invocation**: Executes enrichment step, hits wait, suspends +2. **Second Invocation** (~2 seconds later): Resumes from checkpoint, skips enrichment (uses stored result), completes finalization + +You'll notice: +- Multiple Lambda invocations for a single workflow +- Enrichment step result is reused (not re-executed) +- Total execution time includes the 2-second wait, but you only pay for active compute time + +## Cleanup + +To remove all resources: + +```bash +sam delete --stack-name STACK_NAME +``` + +Or via CloudFormation: + +```bash +aws cloudformation delete-stack --stack-name STACK_NAME +``` + +## Additional Information + +### Key Features Demonstrated + +1. **Checkpointed Steps** +```javascript +const enrichmentResult = await context.step('enrich-order', async () => { + return await invokeNodejsLambda(nodejsLambdaArn, orderId, logger); +}); +``` + +2. **Durable Wait** +```javascript +await context.wait({ seconds: 2 }); +``` + +3. **Structured Logging** +```javascript +logger.info('Starting durable order processing', { + event, + remainingTimeMs: context.getRemainingTimeInMillis?.() +}); +``` + +4. **Error Handling** +```javascript +try { + // Workflow logic +} catch (error) { + logger.error('Order processing failed', { + error: error.message, + errorName: error.name + }); + return { success: false, error: { ... } }; +} +``` + +### Customization Options + +**Modify Wait Duration:** +```javascript +// Wait for 5 minutes +await context.wait({ minutes: 5 }); + +// Wait for 1 hour +await context.wait({ hours: 1 }); + +// Wait for 2 days +await context.wait({ days: 2 }); +``` + +**Add More Steps:** +```javascript +const step1Result = await context.step('step-1', async () => { + // Your logic here +}); + +const step2Result = await context.step('step-2', async () => { + // Your logic here +}); +``` + +**Configure Retry Behavior:** +```javascript +const result = await context.step('my-step', async () => { + // Your logic +}, { + maxAttempts: 3, + backoffRate: 2.0, + intervalSeconds: 1 +}); +``` + +### Supported Regions + +AWS Lambda Durable Functions are available in select regions. Check the [official documentation](https://docs.aws.amazon.com/lambda/latest/dg/durable-supported-runtimes.html) for the latest list. + +As of February 2026, supported regions include: +- us-east-1 (N. Virginia) +- us-west-2 (Oregon) +- eu-west-1 (Ireland) +- ap-southeast-1 (Singapore) + +### Troubleshooting + +**"InvalidParameterValueException: You cannot invoke a durable function using an unqualified ARN"** +- Solution: Always use a qualified ARN (with version or alias). This template automatically creates a `prod` alias. + +**"Cannot find module '@aws/durable-execution-sdk-js'"** +- Solution: Ensure dependencies are installed. SAM CLI automatically runs `npm install` during build. + +**Function times out** +- Solution: Increase the timeout in `template.yaml`: +```yaml +Timeout: 900 # 15 minutes +``` + +---- +Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved. + +SPDX-License-Identifier: MIT-0 diff --git a/lambda-durable-functions-nodejs-sam/example-pattern.json b/lambda-durable-functions-nodejs-sam/example-pattern.json new file mode 100644 index 000000000..c75186b90 --- /dev/null +++ b/lambda-durable-functions-nodejs-sam/example-pattern.json @@ -0,0 +1,63 @@ +{ + "title": "AWS Lambda Durable Functions with Node.js", + "description": "Demonstrates AWS Lambda Durable Functions using Node.js with automatic checkpointing, durable waits, and fault tolerance for long-running workflows.", + "language": "Node.js", + "level": "200", + "framework": "SAM", + "introBox": { + "headline": "How it works", + "text": [ + "This pattern demonstrates AWS Lambda Durable Functions using Node.js to build resilient, long-running workflows that can execute for up to one year.", + "The orchestrator function uses the @aws/durable-execution-sdk-js to implement checkpointed steps with context.step(), durable waits with context.wait(), and automatic recovery from failures.", + "The workflow validates input, invokes an enrichment Lambda function, waits 2 seconds (without compute charges), and finalizes the order processing." + ] + }, + "gitHub": { + "template": { + "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/lambda-durable-functions-nodejs-sam", + "templateURL": "serverless-patterns/lambda-durable-functions-nodejs-sam", + "projectFolder": "lambda-durable-functions-nodejs-sam", + "templateFile": "template.yaml" + } + }, + "resources": { + "bullets": [ + { + "text": "AWS Lambda Durable Functions Documentation", + "link": "https://docs.aws.amazon.com/lambda/latest/dg/durable-functions.html" + }, + { + "text": "Durable Execution SDK for JavaScript", + "link": "https://github.com/aws/aws-durable-execution-sdk-js" + }, + { + "text": "AWS SAM Documentation", + "link": "https://docs.aws.amazon.com/serverless-application-model/" + } + ] + }, + "deploy": { + "text": [ + "sam build", + "sam deploy --guided" + ] + }, + "testing": { + "text": [ + "See the GitHub repo for detailed testing instructions." + ] + }, + "cleanup": { + "text": [ + "Delete the stack: sam delete --stack-name durable-functions-demo" + ] + }, + "authors": [ + { + "name": "Sangeetha S", + "image": "", + "bio": "Technical Account Manager @ AWS", + "linkedin": "sangita-sethumadhavan" + } + ] +} diff --git a/lambda-durable-functions-nodejs-sam/src/enrichment/index.js b/lambda-durable-functions-nodejs-sam/src/enrichment/index.js new file mode 100644 index 000000000..5e91108d4 --- /dev/null +++ b/lambda-durable-functions-nodejs-sam/src/enrichment/index.js @@ -0,0 +1,13 @@ +exports.handler = async (event) => { + console.log('Received order:', event.orderId); + + // Simple enrichment + return { + statusCode: 200, + orderId: event.orderId, + enrichedData: { + customerId: 'CUST-' + Math.floor(Math.random() * 10000), + timestamp: new Date().toISOString() + } + }; +}; diff --git a/lambda-durable-functions-nodejs-sam/src/enrichment/package.json b/lambda-durable-functions-nodejs-sam/src/enrichment/package.json new file mode 100644 index 000000000..e2c8ffd25 --- /dev/null +++ b/lambda-durable-functions-nodejs-sam/src/enrichment/package.json @@ -0,0 +1,12 @@ +{ + "name": "order-enrichment-lambda", + "version": "1.0.0", + "description": "Simple order enrichment service", + "main": "index.js", + "keywords": [ + "aws", + "lambda" + ], + "author": "", + "license": "MIT" +} diff --git a/lambda-durable-functions-nodejs-sam/src/orchestrator/index.js b/lambda-durable-functions-nodejs-sam/src/orchestrator/index.js new file mode 100644 index 000000000..0f5ae249d --- /dev/null +++ b/lambda-durable-functions-nodejs-sam/src/orchestrator/index.js @@ -0,0 +1,163 @@ +const { LambdaClient, InvokeCommand } = require('@aws-sdk/client-lambda'); +const { withDurableExecution } = require('@aws/durable-execution-sdk-js'); + +const lambdaClient = new LambdaClient({ + maxAttempts: 3, + retryMode: 'adaptive' +}); + +class Logger { + constructor(context) { + this.requestId = context?.awsRequestId || 'unknown'; + this.functionName = context?.functionName || 'unknown'; + } + + log(level, message, metadata = {}) { + const logEntry = { + timestamp: new Date().toISOString(), + level, + requestId: this.requestId, + functionName: this.functionName, + message, + ...metadata + }; + console.log(JSON.stringify(logEntry)); + } + + info(message, metadata) { this.log('INFO', message, metadata); } + error(message, metadata) { this.log('ERROR', message, metadata); } + warn(message, metadata) { this.log('WARN', message, metadata); } +} + +class EnrichmentError extends Error { + constructor(message, orderId, cause) { + super(message); + this.name = 'EnrichmentError'; + this.orderId = orderId; + this.cause = cause; + } +} + +function validateEvent(event, logger) { + if (!event) { + throw new Error('Event object is null or undefined'); + } + + if (!event.orderId) { + logger.warn('Missing orderId in event, using default', { event }); + return { orderId: 'ORDER-001', nodejsLambdaArn: event.nodejsLambdaArn }; + } + + if (!event.nodejsLambdaArn) { + throw new Error('nodejsLambdaArn is required in event payload'); + } + + return { orderId: event.orderId, nodejsLambdaArn: event.nodejsLambdaArn }; +} + +async function invokeNodejsLambda(lambdaArn, orderId, logger) { + logger.info('Invoking Node.js Lambda for enrichment', { lambdaArn, orderId }); + + try { + const command = new InvokeCommand({ + FunctionName: lambdaArn, + InvocationType: 'RequestResponse', + Payload: JSON.stringify({ orderId }) + }); + + const response = await lambdaClient.send(command); + + if (response.FunctionError) { + const errorPayload = JSON.parse(Buffer.from(response.Payload).toString()); + logger.error('Node.js Lambda returned error', { orderId, functionError: response.FunctionError, errorPayload }); + throw new EnrichmentError(`Lambda function error: ${response.FunctionError}`, orderId, errorPayload); + } + + const result = JSON.parse(Buffer.from(response.Payload).toString()); + logger.info('Node.js Lambda invocation successful', { orderId, statusCode: result.statusCode }); + return result; + } catch (error) { + if (error instanceof EnrichmentError) throw error; + logger.error('Failed to invoke Node.js Lambda', { orderId, error: error.message, stack: error.stack }); + throw new EnrichmentError('Failed to invoke enrichment Lambda', orderId, error); + } +} + +function finalizeOrder(orderId, enrichmentResult, logger) { + logger.info('Finalizing order with enrichment data', { orderId, hasEnrichmentData: !!enrichmentResult }); + + try { + if (!enrichmentResult || enrichmentResult.statusCode !== 200) { + logger.warn('Enrichment result invalid or incomplete', { orderId, enrichmentResult }); + } + + const finalResult = { + orderId, + status: 'COMPLETED', + enrichedData: enrichmentResult, + finalizedAt: new Date().toISOString(), + message: 'Order finalized successfully' + }; + + logger.info('Order finalization complete', { orderId, status: finalResult.status }); + return finalResult; + } catch (error) { + logger.error('Error during order finalization', { orderId, error: error.message, stack: error.stack }); + return { + orderId, + status: 'PARTIALLY_COMPLETED', + enrichedData: enrichmentResult, + finalizedAt: new Date().toISOString(), + message: 'Order finalized with warnings', + error: error.message + }; + } +} + +async function handler(event, context) { + const logger = new Logger(context); + + logger.info('Starting durable order processing', { event, remainingTimeMs: context.getRemainingTimeInMillis?.() }); + + try { + const { orderId, nodejsLambdaArn } = validateEvent(event, logger); + logger.info('Event validation successful', { orderId }); + + logger.info('Executing enrichment step', { orderId }); + const enrichmentResult = await context.step('enrich-order', async () => { + return await invokeNodejsLambda(nodejsLambdaArn, orderId, logger); + }); + logger.info('Enrichment step completed', { orderId, statusCode: enrichmentResult?.statusCode }); + + logger.info('Waiting 2 seconds', { orderId }); + await context.wait({ seconds: 2 }); + logger.info('Wait completed, continuing execution', { orderId }); + + logger.info('Executing finalization step', { orderId }); + const finalResult = await context.step('finalize-order', async () => { + return finalizeOrder(orderId, enrichmentResult, logger); + }); + + logger.info('Order processing complete', { orderId, finalStatus: finalResult.status }); + + return { + success: true, + orderId, + enrichmentResult, + finalResult, + message: 'Order processed successfully with durable execution', + processedAt: new Date().toISOString() + }; + + } catch (error) { + logger.error('Order processing failed', { error: error.message, errorName: error.name, stack: error.stack, orderId: error.orderId }); + return { + success: false, + error: { name: error.name, message: error.message, orderId: error.orderId }, + message: 'Order processing failed', + failedAt: new Date().toISOString() + }; + } +} + +exports.handler = withDurableExecution(handler); diff --git a/lambda-durable-functions-nodejs-sam/src/orchestrator/package.json b/lambda-durable-functions-nodejs-sam/src/orchestrator/package.json new file mode 100644 index 000000000..59199cab2 --- /dev/null +++ b/lambda-durable-functions-nodejs-sam/src/orchestrator/package.json @@ -0,0 +1,21 @@ +{ + "name": "nodejs-durable-order-processor", + "version": "1.0.0", + "description": "AWS Lambda Durable Functions - Node.js Orchestrator with structured logging and error handling", + "main": "index.js", + "scripts": { + "test": "echo \"Error: no test specified\" && exit 1" + }, + "keywords": [ + "aws", + "lambda", + "durable", + "orchestrator" + ], + "author": "", + "license": "MIT", + "dependencies": { + "@aws-sdk/client-lambda": "^3.700.0", + "@aws/durable-execution-sdk-js": "^1.0.0" + } +} diff --git a/lambda-durable-functions-nodejs-sam/template.yaml b/lambda-durable-functions-nodejs-sam/template.yaml new file mode 100644 index 000000000..d90f3e98f --- /dev/null +++ b/lambda-durable-functions-nodejs-sam/template.yaml @@ -0,0 +1,112 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 +Description: AWS Lambda Durable Functions - Node.js Order Processing Demo + +Globals: + Function: + Runtime: nodejs22.x + Timeout: 900 + MemorySize: 512 + Architectures: + - x86_64 + LoggingConfig: + LogFormat: JSON + ApplicationLogLevel: INFO + SystemLogLevel: INFO + +Resources: + # IAM Role for Lambda Functions + LambdaDurableExecutionRole: + Type: AWS::IAM::Role + Properties: + RoleName: !Sub '${AWS::StackName}-LambdaDurableExecutionRole' + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: lambda.amazonaws.com + Action: sts:AssumeRole + ManagedPolicyArns: + - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole + Policies: + - PolicyName: LambdaInvokePolicy + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - lambda:InvokeFunction + Resource: '*' + - PolicyName: DurableExecutionPolicy + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - states:StartExecution + - states:DescribeExecution + - states:StopExecution + - states:GetExecutionHistory + Resource: '*' + - Effect: Allow + Action: + - logs:CreateLogGroup + - logs:CreateLogStream + - logs:PutLogEvents + Resource: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/*' + + # Enrichment Lambda Function (Simple, Non-Durable) + OrderEnricherFunction: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Sub '${AWS::StackName}-OrderEnricher' + CodeUri: src/enrichment/ + Handler: index.handler + Description: Simple order enrichment service + Role: !GetAtt LambdaDurableExecutionRole.Arn + Timeout: 30 + MemorySize: 256 + + # Orchestrator Lambda Function (Durable) + DurableOrderProcessorFunction: + Type: AWS::Serverless::Function + Properties: + FunctionName: !Sub '${AWS::StackName}-DurableOrderProcessor' + CodeUri: src/orchestrator/ + Handler: index.handler + Description: Node.js Durable Order Processor with structured logging + Role: !GetAtt LambdaDurableExecutionRole.Arn + Environment: + Variables: + ENRICHMENT_FUNCTION_ARN: !GetAtt OrderEnricherFunction.Arn + DurableConfig: + ExecutionTimeout: 120 + RetentionPeriodInDays: 7 + AutoPublishAlias: prod + +Outputs: + DurableOrderProcessorArn: + Description: ARN of the Durable Order Processor Function (use with :prod alias) + Value: !Sub '${DurableOrderProcessorFunction.Arn}:prod' + Export: + Name: !Sub '${AWS::StackName}-DurableOrderProcessorArn' + + OrderEnricherArn: + Description: ARN of the Order Enricher Function + Value: !GetAtt OrderEnricherFunction.Arn + Export: + Name: !Sub '${AWS::StackName}-OrderEnricherArn' + + TestCommand: + Description: Command to test the durable function + Value: !Sub | + aws lambda invoke \ + --function-name ${DurableOrderProcessorFunction.Arn}:prod \ + --payload '{"orderId":"ORDER-123","nodejsLambdaArn":"${OrderEnricherFunction.Arn}"}' \ + --cli-binary-format raw-in-base64-out \ + response.json && cat response.json | jq . + + LogsCommand: + Description: Command to view logs + Value: !Sub 'aws logs tail /aws/lambda/${DurableOrderProcessorFunction} --follow'