Serverless Security: Best practices and mitigation strategies (re:Inforce 2019)

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Serverless Security:
Best practices and mitigation strategies
Jeremy Daly
Chief Technology Officer
AlertMe.news
D E V 1 2

Agenda
The Serverless Security Model
Serverless Risks & Common Attack Vectors
Event Injection
IAM Roles & Permissions
Understanding Serverless Scalability
Best Practice & Mitigation Techniques

About Me
• CTO at AlertMe.news
• Consult with companies building in the cloud
• 20+ year veteran of technology startups
• Started working with AWS in 2009
• Blogger, open-source contributor, speaker
• Publish the Off-by-none serverless newsletter
• Host of the Serverless Chats podcast

Shared Responsibility Model

Shared Responsibility Model for Serverless
AWS
Responsible for security
“of” the cloud
REGIONS AVAILABILITY ZONES EDGE LOCATIONS
COMPUTE STORAGE DATABASE NETWORK
OPERATING SYSTEM + VIRTUAL MACHINES + CONTAINERS
APPLICATION
OWNER
Responsible for security
“in” the cloud
APPLICATIONS (FUNCTIONS)
IDENTITY & ACCESS MANAGEMENT CLOUD SERVICES CONFIGURATION
CLIENT-SIDE DATA IN CLOUD DATA IN TRANSIT

Lambda Runtime Environment
• Control Plane
• Provides function management APIs (CreateFunction, UpdateFunctionCode)
• Manages integrations with all AWS services
• Data Plane
• Controls the Invoke API that runs Lambda functions
• Allocates execution environments to functions
• Chooses an existing execution environment that has already been set up for that function
• Runs the function code in that environment

Execution Environments & MicroVMs
• Dedicated execution environment is used for the lifetime of the function and
then destroyed
• Each execution environment hosts one concurrent invocation, but is reused in
place across multiple serial invocations of the same function
• Execution environments run on hardware virtualized virtual machines
(microVMs)
• MicroVMs are dedicated to an AWS account, but can be reused by execution
environments across functions within an account
• Execution environments are never shared across functions, and microVMs are
never shared across AWS accounts

Comparison of EC2 and Firecracker models for Lambda

IaaS
CLOUD PROVIDER RESPONSIBILITY
• Physical infrastructure, access restrictions to physical perimeter and hardware
• Secure configuration of infrastructure devices and systems
CUSTOMER RESPONSIBILITY
• Regularly testing the security of all systems/processes (OS, services)
• Identification & authentication of access to systems (OS, services)
• Patching and fixing flaws in OS
• Hardening OS and services
• Protecting all systems against malware and backdoors
• Patching and fixing flaws in runtime environment and related software packages
• Exploit prevention & memory protection
• Network segmentation
• Tracking & monitoring all network resources and access
• Installation & maintenance of network firewalls
• Network-layer DoS protection
• Authentication of users
• Authorization controls when accessing application & data
• Log and maintain audit trails of all access to application & data
• Deploy an application layer firewall for event-data inspection
• Detect & fix vulnerabilities in 3rd party dependencies
• Use least-privileged IAM roles & permissions
• Enforce legitimate application behavior
• Data leak prevention
• Scan code & configurations statically during development
• Maintain serverless/cloud asset inventory
• Remove obsolete/unused cloud services & functions
• Continuously monitor errors & security incidents
Serverless
CLOUD PROVIDER RESPONSIBILITY
• Physical infrastructure, access restrictions to physical perimeter and hardware
• Secure configuration of infrastructure devices and systems
• Regularly testing the security of all systems/processes (OS, services)
• Identification & authentication of access to systems (OS, services)
• Patching and fixing flaws in OS
• Hardening OS and services
• Protecting all systems against malware and backdoors
• Patching and fixing flaws in runtime environment and related software packages
• Exploit prevention & memory protection
• Network segmentation
• Tracking & monitoring all network resources and access
• Installation & maintenance of network firewalls
• Network-layer DoS protection
CUSTOMER RESPONSIBILITY
• Authentication of users
• Authorization controls when accessing application & data
• Log and maintain audit trails of all access to application & data
• Deploy an application layer firewall for event-data inspection
• Detect & fix vulnerabilities in 3rd party dependencies
• Use least-privileged IAM roles & permissions
• Enforce legitimate application behavior
• Data leak prevention
• Scan code & configurations statically during development
• Maintain serverless/cloud asset inventory
• Remove obsolete/unused cloud services & functions
• Continuously monitor errors & security incidents
Credit: Ory Segal (@orysegal)

IaaS vs Serverless Security Responsibilities
Credit: Ory Segal (@orysegal)

The 12 Most Critical Risks for ServerlessApplications
• SAS-1: Function event-data injection
• SAS-2: Broken authentication
• SAS-3: Insecure serverless deployment configuration
• SAS-4: Over-privileged function permissions and roles
• SAS-5: Inadequate function monitoring and logging
• SAS-6: Insecure third-party dependencies
• SAS-7: Insecure application secrets storage
• SAS-8: Denial of service & financial resource exhaustion
• SAS-9: Serverless business logic manipulation
• SAS-10: Improper exception handling and verbose error messages
• SAS-11: Obsolete functions, cloud resources and event triggers
• SAS-12: Cross-execution data persistency
By the Cloud Security
Alliance (CSA) and PureSec

Standard Serverless Workflow
LAMBDA
CODE REPOSITORY
EVENT SOURCES
…
CLOUD RESOURCES
Code Deploy
Event Trigger
Interactions
Output

ServerlessAttack Surfaces
EVENT
SOURCES
LAMBDA
CLOUD
RESOURCES
CODE
REPOSITORY
Event Injection
Unauthorized Deployment
Data Tampering
Dependency
Poisoning

Possible Repercussions
• Compromise Data
• Abuse Business Logic
• Bypass Authentication
• Leak Secrets
• Denial of Service (DoS)
• Remote Code Execution

Traditional Security Solutions with Serverless
• WAFs (web application firewalls)
• RASPs (runtime application self-protection)
• EPPs (endpoint protection platforms)
• WSGs (web security gateways)
• IPS (intrusion prevention systems)
• NG-FW (next-generation firewalls)
Your app security is based on GOOD CODING
and STRICT CONFIGURATION
❌
❌
❌
❌
✅*
❌

47+ supported event sources that can trigger Lambda
• Amazon S3
• Amazon DynamoDB
• Amazon Kinesis Data Streams
• Amazon Simple Notification
Service (SNS)
• Amazon Simple Email Service
• Amazon CloudWatch Logs
• Amazon CloudWatch Events
(as a proxy to 25+ other services)
• Scheduled Events
• AWS Config
• Amazon Alexa
• Amazon Lex
• Amazon API Gateway
• AWS IoT Button
• Amazon CloudFront
• Amazon Kinesis Data Firehose
• Amazon Simple Queue Service
(SQS)
*Many different event formats*

CloudWatch Logs and Kinesis Data Streams Events
{
"awslogs": {
"data":
"H4sIAAAAAAAAAHWPwQqCQBCGX0Xm7EFtK+smZBEUgXoLCdMhFtKV3akI8d0bLYmibvPPN3wz00CJxmQnTO41whw
WQRIctmEcB6sQbFC3CjW3XW8kxpOpP+OC22d1Wml1qZkQGtoMsScxaczKN3plG8zlaHIta5KqWsozoTYw3/djzwh
pLwivWFGHGpAFe7DL68JlBUk+l7KSN7tCOEJ4M3/qOI49vMHj+zCKdlFqLaU2ZHV2a4Ct/an0/ivdX8oYc1UVX86
0fQDQiMdxRQEAAA=="
}
}
This data has to be decoded, unzipped, and then inspected to
make sure it’s safe to use.

An S3 Example
{
"Records": [
{
"eventSource": "aws:s3",
"eventName": "ObjectCreated:Put",
"s3": {
"bucket": {
...
},
"object": {
"key": "1%22%29%3B%28delete+*+from+uploads",
"size": 4
}
}
}
]
}
"1%22%29%3B%28delete+*+from+uploads”,

Trust No One 👽
let filename = decodeURIComponent(s3.object.key.replace(/+/g,'%20'))
connection.query(
'INSERT INTO uploads (`file`) VALUES ("' + filename + '")',
(error, results) => {}
)
INSERT INTO uploads (`file`) VALUES ("1");(delete * from uploads)
Even if you’ve never done this, one of your developers will!!!
$♂

“Every program and every privileged user of the
system should operate using the least amount of
privilege necessary to complete the job.”
Jerome Saltzer
Communications of the ACM

Least-Privileged IAM Roles
• Functions should only be allowed to do
what they need to do
• AWS IAM model is extremely powerful,
but easy to get wrong
• Human factor (laziness, ignorance)
• “Over-privileged” issues are likely the most
common mistake in serverless applications
BatchGetItem
BatchWriteItem
CreateTable
DeleteItem
DeleteTable
DescribeLimits
DescribeReservedCapacity
DescribeReservedCapacityOfferings
PurchaseReservedCapacityOfferings
DescribeStream
DescribeTable
GetItem
GetRecords
GetShardIterator
ListStreams
ListTables
ListTagsOfResource
Query
Scan
TagResource
UntagResource
UpdateItem
UpdateTable
PutItem

Limiting IAM Permissions
• Use a “role-per-function” model
• Use SAM managed policies
• Serverless Framework: use custom roles per
function or the “serverless-iam-roles-per-
function” plugin
Minimize the blast radius of vulnerable functions

Scaling ServerlessApplications
• Serverless apps are more resilient to traffic spikes
and can scale to support very high bandwidth
• Synchronous vs. Asynchronous invocations
• Invocation type is pre-determined for each service type
• Examples of DoS (or Denial of Wallet):
• Synchronous: flood with API Gateway requests
• Asynchronous: flood with S3 files
• Poll-based / Stream-based: send malformed batch of events to the stream
• Poll-based / Not stream-based: queue message retention can be up to 4 days for SQS

Best Practices & Mitigation Techniques
• Use API Gateway ‘quota’ and ‘throttling’ capabilities
• Consider using API response caching
• Use SQS as a broker
• Set up Dead Letter Queues (DLQs)
• Design for retry
• Define reserved capacity limit per function
• Set timeouts to avoid “hangs” on unexpected input

Best Practices & Mitigation Techniques (continued)
• Use API Gateway Lambda Authorizers
• Protect your keys, usernames and passwords
• Monitor your concurrent executions, throttling metrics, errors and timeouts
• Set up billing alerts
• Delete old functions, triggers, and resources

Thank you!
Jeremy Daly
@jeremy_daly
jeremydaly.com

Serverless Security: Best practices and mitigation strategies (re:Inforce 2019)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Serverless Security: Best practices and mitigation strategies (re:Inforce 2019)

Similar to Serverless Security: Best practices and mitigation strategies (re:Inforce 2019) (20)

Recently uploaded

Recently uploaded (20)

Serverless Security: Best practices and mitigation strategies (re:Inforce 2019)