Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SQS event integration #5074

Merged
merged 4 commits into from Jul 3, 2018
Merged

Add SQS event integration #5074

merged 4 commits into from Jul 3, 2018

Conversation

alexdebrie
Copy link
Contributor

@alexdebrie alexdebrie commented Jun 25, 2018

What did you implement:

Adds SQS event integration 馃帀

How did you implement it:

EDIT 6/28: SQS integration is live! Full testing instructions below.

I saw Brett Andrews note that Lambda + SQS integration is coming soon. Then I peeked at the SAM implementation of the SQS event. It looks to be very similar to Kinesis / DynamoDB streams as it's a pull-based integration. Like the streams, you can set batch size. There is no need for the startingPosition parameter though.

I found the necessary IAM permissions by looking at the permissions on this role.

How can we verify it:

We need to wait until the integration is actually released. Once it is, you could use the following serverless.yml to test:

# serverless.yml

service: sqs

provider:
  name: aws
  runtime: nodejs6.10

functions:
  hello:
    handler: handler.hello
    events:
      - sqs:
          arn:
            Fn::GetAtt:
              - MyQueue
              - Arn

resources:
  Resources:
    MyQueue:
      Type: "AWS::SQS::Queue"
      Properties:
        QueueName: "MyQueue"

After you deploy, run the following commands with the awscli to put a message into your Queue:

$ QUEUE_URL=`aws sqs get-queue-url --queue-name MyQueue --query 'QueueUrl' --output text`
$ aws sqs send-message --queue-url ${QUEUE_URL} --message-body "test"

Then check to make sure your Lambda fired:

$ sls metrics
Service wide metrics
June 27, 2018 3:16 PM - June 28, 2018 3:16 PM

Invocations: 1
Throttles: 0
Errors: 0
Duration (avg.): 3.63ms

Todos:

  • Write tests
  • Write documentation
  • Fix linting errors
  • Make sure code coverage hasn't dropped
  • Provide verification config / commands / resources
  • Enable "Allow edits from maintainers" for this PR
  • Update the messages below

Is this ready for review?: YES
Is it a breaking change?: NO

@alexdebrie alexdebrie changed the title [WIP] Add SQS event integration Add SQS event integration Jun 28, 2018
@@ -46,6 +46,7 @@
"./aws/package/compile/events/cloudWatchEvent/index.js",
"./aws/package/compile/events/cloudWatchLog/index.js",
"./aws/package/compile/events/cognitoUserPool/index.js",
"./aws/package/compile/events/sqs/index.js",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering why this plugin wasn't running!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃檲 Yea, took me a while to find this one 馃榿

events:
- sqs:
arn: arn:aws:sqs:region:XXXXXX:myQueue
batchSize: 100
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10 is the maximum batch size allowed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Updated.

@horike37 horike37 added this to the 1.28 milestone Jun 28, 2018
@domaslasauskas
Copy link

It's here! 馃帀 AWS Lambda Adds Amazon Simple Queue Service to Supported Event Sources

Copy link
Member

@horike37 horike37 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexdebrie
Great to see working fine 馃帀 馃帀 馃帀
LGTM

@horike37 horike37 merged commit cbc5e3c into master Jul 3, 2018
@horike37 horike37 deleted the SQS branch July 3, 2018 00:54
@pmuens
Copy link
Contributor

pmuens commented Jul 3, 2018

This is awesome! Thanks for implementing this @alexdebrie 馃憤 馃帀

@QAnders
Copy link

QAnders commented Jul 10, 2018

Any plans to implement the "Extended Client" functionality as in AWS Java SDK to support messages larger than 256kb by storing the payload/body on S3?

E.g.:

resources:
  Resources:
    MyQueue:
      Type: "AWS::SQS::Queue"
      Properties:
        ExtendedLargeMsg: "MyS3Bucket"
        QueueName: "MyQueue"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants