Using Kotlin in a Serverless Architecture with AWS Lambda — Part 3: Designing and Implementing a Multi-Tier Lambda App

High level reference architecture with tools and products that you may need

Juan Urrego
Tech@Travelstart

--

In part 1 and 2 of this series, we have covered the basics to set up our serverless project and development environment using emulation software. Now that we already understand the basics and tools, let’s go in deep with our architecture.

Index

  1. Setting up the project
  2. Setting up your local development environment
  3. Designing and implementing a Multi-Tier Lambda App
  4. A Multi-layer core for your function: A request-dispatcher and micro HTTP router implementation
  5. Testing your code
  6. Integrating a RDBMS to your project (coming soon)
  7. Authentication & Authorisation using Amazon Cognito (coming soon)
  8. Creating an end-to-end RESTFul service (coming soon)

Old Fashion Design Patterns Still Work

In the article “Silence of the Lambdas”, Dan Greene states that we shouldn’t stop regular best practices just because it’s serverless. The fact that we no longer need to maintain and setup servers, doesn’t mean that we stop thinking about quality attributes: performance, security, modifiability, etc.

Therefore, let’s take a look to a possible reference architecture that can help us in future serverless-based projects, as well as the tools and design patterns to achieve that.

N-Tier Architecture

A multi-tier or n-tier pattern is a way to distribute and group our logical components along our deployment nodes for the sake of an independent execution structure. In an N-Tier architecture the logical layers play a critical role, given that each layer, as a unit of implementation, separate responsibilities and manage dependencies of common functionalities. To give you an example, one tier can host multiple logical layers and, in other cases, there could be layer per tier. The next diagram exposes the previous statement:

Diagram 1: Mono-Tier vs Multi-Tier architecture using a multi-layer pattern

In the previous diagram, the 1-Tier architecture is a typical example of an old fashion desktop app, where we used to have all the logical components deployed in the same execution node. The second case, the 4-tier architecture, it’s a modern case where the visual components and UI logic reside on the client side, while the business logic components are distributed in backend deployment nodes (as well as the data components).

That means that our serverless backend may have multiple layers that are going to be deployed in the same AWS Lambda function, but also we may have other lambda functions that could deploy other layers. One of the common anti-patterns that I have seen in Function As a Service (FaaS) projects, is that developers put all the logic in the handling function without creating a separation, resulting in a single file with thousands of lines of code.

In a multi-layer pattern the most common practice is to use a closed layer architecture, meaning that a layer can only call the next layer immediately down, avoiding cyclic dependencies and improving security. However, some people have found benefit in an open layer architecture, where a layer can call any of the layers below. Personally, this latter tactic makes the source code messy and insecure, so I recommend avoiding it.

Serverless Reference Architecture

Now that we already understand the main architectural pattern to use, let’s take a look at a Serverless Reference Architecture.

Diagram 2: Serverless Basic Web Reference Architecture

As you can see in diagram 2, I’m using a multi-tier architecture for the complete serverless product. In this case we have a 4-tier architecture conformed by the next tiers.

Client Tier

Diagram 3: Client Tier

This tier renders the UI components of our product. Using Client Side Rendering (CSR) via a Single Page Application (SPA) framework just like React, Angular, or Vue, is one of the most common ways to work here. I’m a true believer that client devices are powerful machines that we need to take advantage of to reduce our on demand infrastructure costs and state management that a Server Side Rendering (SSR) generates. In terms of performance and SEO behaviour, there’s still a debate about using CSR or SSR, however I’m not going to go in detail about that.

These visual components need to be retrieved from a distribution server that is located in the next the tier: the presentation tier, so let’s take a look at it.

Presentation Tier

Diagram 4: Presentation Tier

This tier is responsible for exposing the front-end services that our clients will require to perform operations. In this case, I recommend having a Content Delivery Network (CDN) such as Cloudfront to cache and distribute your UI resources (located in our application tier). However, there are some non-AWS solutions such as Cloudflare that can provide not only caching services but also some security services such as Web Application Firewalls (WAF) and security rules to prevent DoS and DDoS attacks (AWS also provides some of them).

In this Tier we can find the API gateway, responsible to expose our API services to our client apps and eventually third party systems. In the case of the AWS API Gateway, the lambda functions will be accessed using events, so even when the client communicates with the API using a request-reply approach, behind the scenes we have an Event Driven Architecture (EDA) to manage our requests.

Finally, we’ll require an authentication & authorisation service for our clients. As usual, you can develop your own, but I recommend to use out-of-the-box solutions that use the best practices, most common frameworks and industry standards. In the case of AWS, we can use Amazon Cognito as our Authentication as a Service (AaaS) provider. Cognito will help us with the complete authentication and authorisation life cycle of our system’s users. There are other non-AWS services such as Auth0 or Back& that you could also explore.

Remember that this tier should be the only Demilitarised Zone (DMZ) of your infrastructure, therefore the following tiers won’t have public access.

Application Tier

Diagram 5: Application Tier

Also called Business Tier, the Application Tier is in charge of grouping all the core components of our architecture, meaning: our functions. A critical concept in this tier, which I’m going to explain in detail in part 4, is the Function as a Microservice (FaaM), where every Lambda function should behave as a small microservice.

In a FaaM project, every communication between our functions should be event driven, meaning that we must reduce as much as possible the request-reply interaction between them, otherwise we could create an unnecessary coupling between our microservices. To achieve this, we can use services such as SQS, SNS or Kinesis.

Diagram 6: Asyncronus communication between lambda functions using a Publisher-Subscriber pattern with SNS

Typically in non-serverless architectures, a Front-end web server would be used as solution to host our UI resources and UI logical components. However, by using SPAs and CSR we can easily host our visual resources in S3 to have a fully serverless architecture. By using a framework like React or Angular it’s just as easy as runningnpm run build and uploading the resulting files to an S3 Bucket. All the UI logic and state will be managed by the client side, thus eliminating the need of a web server.

Note: Don’t forget to create security policies for your AWS Lambda functions and S3 Buckets. In the case of Lambda, you can locate the functions in a private network and secure them creating specific security groups and roles with just enough permissions.

Data Tier

Diagram 7: Data Tier

The data tier is responsible of hosting all the services that will persist and manage our data (OLTP databases, OLAP databases, caches, etc). Diagram 7 it’s showing us an RDS database (soon we might be able to replace this with Aurora Serverless) using a multi-tenant pattern in bridge: single database with multiple schemas (one per function/microservice), as well as DynamoDB tables for caching and keeping operational states. This topic will be further explained in part 6 of this series, however it’s important to keep in mind that, even when some people embrace having a complete data tier with DynamoDB, the answer will be “it depends”.

DynamoDB is a Key-Value store useful for a specific use case set, such as caching and keeping state via simple queries execution. In contrast, the relational databases were designed to satisfy classic transactional use cases. Each database type has pros and cons, so our design should be driven by these tradeoffs and not by how cool or serverless the solution is. Forrest Brazzeal illustrates the most common pitfalls of using DynamoDB and explains in detail when to use it or not. The next image presents a decision tree for DynamoDB.

How to decide when DynamoDB is for you. ©Forrest Brazzeal from link

Note: If the no-SQL boom of the 2010’s taught us anything, is that relational databases still a good (maybe the best) option for our daily transactional operations.

Support Area

Diagram 8: Support Area

The support area contains Cloudformation S3 buckets, IAM policies, Route 53 Hosted Zones, Cloudwatch, among others. In general, it audits, logs and monitors the actions of the other tiers. We can use S3 buckets to persist structured log files as well as Cloudwatch to monitor our lambda functions and databases efficiency. Part 8 will explain this tier in depth, as well how to configure/implement it.

Final Thoughts

This article presented a basic high-level web reference architecture for a serverless project. Based on that, it’s important to take into consideration the next points:

  • The data tier is not necessarily 100% serverless and we need to understand the tradeoffs that those decisions might have.
  • New technologies and trends shouldn’t replace/avoid the basics of software engineering and software architecture.
  • Architectural and design patterns shouldn’t be forgotten.
  • Keep your functions communication as asynchronous as possible using EDA patterns.

The next articles will explain in detail the application and data tiers. For that reason, the client and presentation tiers are out of the scope of this series. If you need help with those tiers, there are other articles that can help you.

See you soon and don’t forget to clap & subscribe.

Part 4: A Multi-layer core for your function: A request-dispatcher and micro HTTP router implementation

--

--