How to Save JSON Data from an ArcGIS Online Webhook to DynamoDB Using AWS Lambda

MasaakiKurokawa — Tue, 23 Jul 2024 12:03:25 GMT

Here is an example code for saving JSON data received from an ArcGIS Online webhook to DynamoDB using AWS Lambda.

### Prerequisites

1. **Create a DynamoDB Table**
- Open the DynamoDB service in the AWS Management Console and click "Create table."
- Set the table name (e.g., `WebhookData`) and the primary key (e.g., `id`).

2. **Grant DynamoDB Access to the Lambda Function**
- Add DynamoDB access permissions to the IAM role of the Lambda function.
- Attach the `AmazonDynamoDBFullAccess` policy.
- Attach the `CloudWatchLogsFullAccess` policy for Lambda to verify the operation.

### Lambda Function Code

Paste the following Python code into your Lambda function. This code receives JSON data sent from the webhook and saves it to DynamoDB.

```python
import json
import boto3
import uuid
from datetime import datetime

# Create a DynamoDB client
dynamodb = boto3.resource('dynamodb')
table_name = 'WebhookData'
table = dynamodb.Table(table_name)

def lambda_handler(event, context):
# Receive JSON data sent from the webhook
try:
# You can add any necessary processing here
# For example, extracting specific fields
print("Received event: " + json.dumps(event, indent=2))

# Create data to save in DynamoDB
item = {
'id': str(uuid.uuid4()), # Generate a unique ID
'timestamp': datetime.utcnow().isoformat(), # Add a timestamp
'data': event # The received JSON data
}

# Save data to DynamoDB
table.put_item(Item=item)

return {
'statusCode': 200,
'body': json.dumps('Webhook data saved successfully!')
}
except Exception as e:
print("Error processing webhook: " + str(e))
return {
'statusCode': 500,
'body': json.dumps('Error processing webhook')
}
```

### API Gateway Configuration

Use an already configured API Gateway to allow the Lambda function to receive HTTP requests from external sources.

To enable the Lambda function to receive HTTP requests from external sources, configure API Gateway:
- Open the API Gateway console and create a new HTTP API.
- In the "Resources" tab, create a new resource.
- Create an integration: set up the Lambda function.
- Stage name: `$default`

### Set Up Webhook in ArcGIS Online

Use the endpoint URL obtained from API Gateway to set up the webhook in ArcGIS Online.

### Testing

To test if the webhook is correctly set up, trigger an action in ArcGIS Online that activates the webhook. Check the Lambda function logs in AWS CloudWatch Logs. Also, verify that the data is saved in the DynamoDB table.

To download logs:
1. Go to DynamoDB > Tables > Explore items.
2. Select the desired items, click "Action" > "Export to CSV."
3. The CSV file will be downloaded.

By following these steps, you can save data received from an ArcGIS Online webhook to AWS DynamoDB. You can also process the data and integrate it with other AWS services as needed.

**Tips**: If it doesn't work, check the data flow between API Gateway, Lambda, and DynamoDB. Note that the webhook interval is set to 30 seconds, so the data may take 30 seconds to 1 minute to appear in DynamoDB.

idea How to Save JSON Data from an ArcGIS Online Webhook to DynamoDB Using AWS Lambda in ArcGIS Online Ideas

How to Save JSON Data from an ArcGIS Online Webhook to DynamoDB Using AWS Lambda