1. Help & Support
  2. Hyperglance Admin

Configuring Hyperglance to use S3 for large SNS messages

SNS has a message limit of 256KB. Hyperglance can be configured to place the message JSON in an S3 bucket and post a reference to that object over SNS.

Configuration Steps:

1) Create an S3 bucket in the same account and region as your SNS Topic.

Note: The access policy used to publish to SNS must now also allow: S3:PutObject

2) SSH into the Hyperglance instance/VM

3) Edit file: /var/lib/data/hyperglance/config.env

4) Add a line to the file to reference the name of your S3 bucket:

AWS_SNS_PAYLOAD_BUCKET=the_name_of_your_bucket

4) Save the file

5) Run this command to detect the config change and restart Hyperglance:

sudo docker-compose -f /etc/docker-compose.yml up -d

Results:

When the rule next runs you should now see the JSON payloads in the S3 bucket:

WARNING: These JSON objects will remain the bucket indefinitely. It is your responsibility to either create a lifecycle rule in S3 to auto-delete the object or to have a Lambda function delete the object after it is processed.

 

The JSON payload sent over SNS will now include a "data" sub-object that references the bucket and object key (it will no longer include the "results" list):

{
"name": "Test",
"threshold": 0,
"evaluatedAt": 1606933410023,
"labels": ["Example"],
"status": "ABOVE_THRESHOLD",
"data": {
"s3bucket": "the_name_of_your_bucket",
"key": "ffc85700-e6ab-4be7-9443-1e417110f447_Test_1606933410023.json"
}
}