Write Once, Test Everywhere — Simplified SDK Testing
How PayPal used the Platform Agnostic Automation Framework to simplify SDK testing
The latest and most advanced approaches to the product development lifecycle are constantly trying to achieve the best balance between speed and efficiency. With customer-centric product approaches taking center stage in today’s world, increased speed of product delivery should not compromise efficiency and the quality of the product.
One way to solve that problem is by reducing the redundancies in the software development and testing processes. The Platform Agnostic Automation Framework (PAAF) is uniquely designed to address and resolve that problem by simplifying the SDK (Software Development Kit) testing process. The underlying rationale behind PAAF focuses on writing test cases once and executing them against multiple platforms. Key benefits of utilizing PAAF include ease of maintenance, unified test coverage reports across all platforms, and unified UI (User Interfaces) validations.
Software Development Kit (SDK)
A Software Development Kit (SDK) is a toolkit that allows third-party developers to build applications that satisfy their functional aspects. An SDK is specific to a platform and distributed as an independent library. SDK tools include Libraries, Application Programming Interfaces (APIs), Documentation, and Code Samples or Sample Applications.
Challenges in testing SDKs
Quality is at the core of any successful product, and assuring high quality is always the priority of any development cycle. Well-rounded functional and automation testing aims to deliver a consistent and intended product experience. Testing often becomes challenging and time-consuming with different platforms like iOS, Android, Web, etc.
Traditionally, the practice constitutes developing redundant test cases and test applications for each SDK integrating platform. For example, to test an SDK API sendMoney, the developer or tester must create a test application and write the same test cases on different platforms. This adds overhead to existing testing and automation processes and can delay product releases. Furthermore, when native UI elements are involved, it incurs platform-specific challenges that need to be handled.
Platform Agnostic Automation Framework (PAAF)
PAAF is a unique framework designed to address the core challenge of writing and supporting the same scripts on multiple platforms. PAAF is designed with the goal of writing once and running everywhere.
The PAAF framework allows us to write the test scripts using standard Java Testing Frameworks like Junit/TestNG and run them against any platform. It also allows us to validate the UI elements abstracting the platform-specific challenges. These scripts can be easily integrated with Code Coverage and Continuous Integration while keeping things simple with the familiar and well-oiled goodness of Junit/TestNG.
Note: There are tools available in the market like Appium, but we realized that they are primarily for native applications that are heavily UI-driven.
Big Picture
How does it work?
- The QE (Quality Engineer) writes the test scripts using any Java testing tool (Junit/TestNG), invoking the SDK interface.
- The test framework transforms the SDK API calls to Unified Automation Message (UAM) and sends them to the server.
- The server sends the requests to the target platform configured by the test scripts.
- The actual platform SDK APIs are invoked by the native delegators when they receive the UAMs.
- The test scripts receive the response as UAMs, perform validation, record the status of each test case, and generate reports for platforms.
Code Snippets
The following code snippets show the API to Creating a Transaction using PayPal Here SDK for Android, Web, and iOS.
Android
Objective-C
JavaScript (web)
Test Case using PAAF
The below test case is written using TestNG to Create a Transaction, which is executed against Web, Android, and iOS without any code change.
The code is only dependent on the TestNG framework and the SDK Interfaces.
Line 12 in the below code snippet shows how the SDK API is invoked agnostically.
PAAF in Action
The following video shows a simple test case to accept KeyIn transactions using PayPal Here SDK. The video covers running the same test case against Android and Web platforms. Notice that the platforms can be easily configurable. The video also covers how the UI interactions are managed from the single TestNG test case.
Benefits of PAAF
- Write the test scripts once and run them against all platforms.
- No need for a different test environment setup for each platform.
- Abstracts platform-specific challenges.
- Test scripts do not require any PAAF knowledge.
- Inherits all benefits of Junit/TestNG.
- Seamlessly switches environments and performs comparisons.
- Java-Based coverage tools can generate test Reports and Code Coverage Reports.
We continuously strive to achieve technical excellence, deliver high-quality software, hit targets, and meet go-to-market timelines. Adopting frameworks like PAAF offers one less variable in the continuous product development cycle. It not only unifies the automation process, but also simplifies human practices to achieve steady and predictable results.
Contributors: Biju Balakrishnan Nair, Sreepada Rao, Gantumur Tserevkhuu, and Victor Joseph
