Test Automation Estimation

9 min readOct 19, 2022

In this article, I want to show you how can estimation be done, which techniques can be used and how to prepare yourself for the estimation process in general. But take into account that each project can have specific features or approaches as well.

So, in general, Test Automation activities can be divided into 2 phases.

Initial phase activities

In this phase, no real tests were not written yet. Only mock tests and some UI Page openings in case UI automation happened. Basically, it is the preparation phase.

General onboarding activities — usually 1–2 days to get the first impression of the application
Requirements/ manuals learning — may take up to 1 week. Usually, applications have some guides for complex features how they work and how we should test them. Once I have a project when such guideline was a 300-page book :D
Creating/Updating Test Automation Strategy — usually, you will be asked to update or create Test Automation Strategy. Sometimes it is a brand new artifact in a project, and sometime Test Automation Strategy will be included in Master Test Plan. Depends on the rules in your company. Usually takes 6–8 hours
Set up an environment (optional): Installing libraries using scripts, Setting configuration variables, etc. — This is a procedure to create an environment for automation testing. Sometimes this part will be done by DevOps, but in case not — consider that you may need additional time for this. Also, you may need to set up the ability to run tests on the local environment — in that case, usually, it may take 3–6 hours for cases when developers/DevOps already prepared scripts for setting up all services.
Mocking Component Behavior(optional) — sometimes you may need to write your own mock server or event simulator for some IoT device. For example IoT or Embedded solutions
Set up framework template — make a dry run of the first test, make sure that all dependencies are up to date, add additional required ones, check that logging and reporting work and can be run on your local machine, and check that it is possible to run tests on different envs, browsers, devices — usually takes 4–8 hours
Set up CI/CD pipeline — make sure that tests can be run from CI, the report is generated, the history trendline is shown(in the case of Allure Framework Reporting), make sure that all infrastructure variables are hidden in GitLab envs, for example, make sure that tests can be run on different envs — usually for API tests takes 4–6 hours, in case UI automation also presented — make take up to 8–10 hours additionally.

In-Sprint activities

in this phase in-sprint testing are made. So take into consideration the next activities:

Requirements analysis — usually up to 1–2 hours for each user story/requirement
Manual testing for UI automation cases(optional) — 1–2 hours — to get a better understanding of how the feature works.
Implementing Page Object Model and helpers methods — depends on elements to interact, in case id is not presented may take for each element up to 5mins, and for each helper method( click, fill, clear, move, etc.) 5–10 mins. So in case, the page consists of 2 buttons, and 2 text fields that can have errors — approx estimation will be 6*5 + 6*10 + 1hour for risks = 2.5-3 hours to describe POM with methods. Additionally may take developing common web components (like calendars, some custom drop-downs, etc)
Implementing API clients with helpers methods — for each method usually takes approx 1–2 hours
Implementing Data Transfer Objects / Response Models —depending on the number of fields and complexity of the response object you will be using — usually takes around 1–2 hours
Test Data Preparation — Creating synthetic test data and Creating dynamic & random test data.
Test design — which tests you will automate and how without any concrete tests — 1–2 hours
Test scripts implementation and run– each test takes around 10–30 mins
Code review(optional) — 2–6 hours per user story
Developing helpers functions for Manual QAs(optional) — depends on it — usually, it is some helpers API calls which can be reused from our tests — usually, up to 2 hours
Test Reporting + Test run at the end of the sprint + analysis of test run — 1–2 hours. This may include: Generating summary reports/emails; Linking screenshots/logs to the reports; Triggering emails on test execution, completion, etc; Rerunning failures(It is helpful to automate the re-execution of failure/flaky tests. It also helps to examine the consistency and probability of failure.)
Updating results to ALM(Application Lifecycle Management) Tools : Ex: JIRA, Team Foundation Server, Polarion by using API endpoints or CLI
Cleanup of Environment.
Monitoring Production Logs

Types of estimation techniques in software testing

1. Work Breakdown Structure (WBS) :

Break the work into smaller modules for easy & accurate estimations. In this technique, a testing task is broken down into smaller modules and those modules are further divided into measurable sub-modules. Each sub-module is, in turn, divided into functionalities and they are split into sub-functionalities. The outcome is a very detailed, tightly coupled, traceable yet easy to understand, and manageable hierarchical map of project functionality. Using any other estimation technique, these modules are estimated to get actual effort.

2. 3-Point software testing Estimation:

The statistical way of estimation. Similar to the WBS, it also divides the task into smaller sub-tasks. But, this is a statistical method where three possible scenarios should be estimated for each sub-task based on prior experience or best guesses.

B = the best-case estimate

M = the most likely estimate

W = the worst-case estimate

B. The best-case estimate: It assumes that the project aspects like the presence of a highly skilled team, and availability of necessary resources, everything will go right and the project will face no blockers. Now using any other estimation technique project is estimated for the best case to come to a value. Let’s assume this number is 50 man-days for our further calculation.

M. The most likely estimate: It assumes that when the team is skilled, with enough project resources, most things will go well with very few blockers. Now using any other estimation technique project is estimated for the most likely case to come to a value. Let’s assume this number is 70 man-days for our further calculation.

W. The worst-case estimate: When the team is insufficiently skilled, the project has limited resources and most things will not go as planned. Now using any other estimation technique project is estimated for the worst case to come to a value. Let’s assume this number is 100 man-days for our further calculation.

Estimate Calculations

B, M, and W respectively are used to calculate ‘E’ and ‘SD’.

‘E’ for estimation, and

‘SD’ for standard deviation — measures the variability or uncertainty in the estimate

E= (B+4M+W)/6

For above example, E = (50+(4*70)c+100)/6 = 71.6 man-hours

SD = (W-B)/6

For above example, E = (100–50)/6 = 8.3

Therefore, the nearest approximation for the testing estimate can be considered to vary within the range of E+SD to E-SD man-days. For the above example, the estimate will vary from 63.3 to 79.9 man-hours.

3. Function Point Analysis (FPA) :

Estimation from the functionality standpoint.

Function point analysis is based on the specifications of the project or design. Similar to the WBS, tasks are divided into modules and each model is given a function point, depending on its complexity. Modules with higher complexity have higher points. This technique indicates software functionality from the user’s point of view and estimates the value of Total Effort with respect to time, cost, or size of the task.

Total Effort = Total FP x Estimate per FP

Estimate per FP is defined by the test manager on the basis of team experience and skill, with respect to time, money, or size. For instance, 10hours/points or $100/points. The FP for each module = No. of modules of a certain difficulty x FP for that module. Each module’s FP is then added to have the total FP.

4. Delphi Technique

This technique is a group estimation technique and is one of the most popular where estimates are derived after multiple rounds of questionnaires sent to a panel of experts. It has the following steps -

An expert panel makes forecasts, with reasons, based on the results of multiple rounds of questionnaires regarding how many hours a certain task or project will take under the guidance of the manager.
After the first round, the experts are allowed to revise their estimates based on how they interpret group responses, accounting for other experts’ judgment.
Rounds are repeated till the range of forecasts decrease and an average value is reached.
This method is simple and reliable as the experts are highly experienced in the subject area. The resulting estimates from this technique reflect the consensus estimation of the group of experts.

5. Agile Estimation

In the previous techniques, details and requirements are defined before we plan the schedule and budget. This has some drawbacks because the software industry is constantly changing and hence the use of the previous techniques is decreasing. The test estimation techniques in agile support continuous delivery. In these techniques, the currently available data and prior experience are used for estimation and new information is continuously integrated into the project to refine the process of estimation. Some of the widely used agile estimation techniques are -

Planning Poker: It is a consensus-based technique for estimating, mostly used to estimate the effort or relative size of testing by breaking project work into sprints. At the start of each sprint, estimates are done for the user stories (smallest measurable user requirement) and priorities are defined. The team members use a deck of cards with numbers on them from 0 to 21 in the Fibonacci sequence (0, 1, 2, 3, 5, 8, 13, 21). These numbers represent ‘Story Points’. The meeting moderator describes a story and asks agile team members to estimate the effort privately without consulting any other team member. Estimates are done on the story-pointing scale. Members were then asked to hold up the card at the same time showing the effort they think is required for the story. This consensus repeats with discussion until all the votes are accounted for and reasoned for.
T-Shirt Sizing: Sometimes, the story-pointing scale is overwhelming for the team members to estimate. In such cases it is more efficient to switch to a non-numerical system like T-shirt sizes: XS, S, M, L, XL, and so on, with these sizes corresponding to the story size that the member estimates a story to have. This presents a simple yet accurate way to estimate testing efforts.

6. Test Case Point Analysis / Test case based estimation

This test case based estimation technique is useful when test case writing is completed or a number of test cases and their complexity is known or estimated beforehand. In the Test Case Point (TCP) analysis, the test cases are used as input for estimating testing efforts. Test cases are classified in terms of complexity. Usually as low, medium, high, and very high. Then considering a test case of each complexity level, an effort value can be estimated for each level of complexity. This value can also be measured by running a test case each from the complexity levels and noting the time it took to run the test. Then this time is multiplied by the number of test cases of each category to come to final estimates of the complete test case set. For eg, assume we have to estimate the testing effort of a test case set of 100 test cases

Step 1. Classify test cases on the complexity scale.

Step 2. Estimate the time it will take to run test cases for each complexity level.

Step 3. Calculate the total estimates for running all test cases using numbers from Step 1 and Step 2.

7. Estimation based on Historical Data:

This method analyses and studies the historical data from past testing projects. This technique works on the rule that time taken for testing projects in the past will take similar efforts for similar complexity projects or functionality. This method is useful for estimating projects which have similar nature, technical stack, and test team members.