The Tester’s Dilemma: Manual v/s Automated Testing

1

Key Players and Resources

Software Engineers/Testers and Software

Testers and Automation Tools

2

Execution of Test Cases

Test cases are executed manually by Testers using guidelines, checklists, and strict processes to write test cases.

Test cases are executed using automation tools and facilitators like data drives, keywords, etc. that accelerate the testing process.

3

Speed

Manual Testing is a time-consuming, long-drawn process.

Automated Testing is much faster and can even be as fast as 5 times the speed of Manual Testing.

4

Accuracy

Manual Testing is prone to human error in writing test cases and test reports.

Automated Testing is more accurate as approved test scripts are automatically executed without human intervention.

5

Go-to-Market Time

In Manual Testing, it takes much longer for the app to be ready and this is not compatible with the volatile digital age.

Go-to-Market time can reduce by up to 15% under Automated Testing, giving a sharp edge over competitors.

6

Cost of Capital

Capital Investment is not expensive for Manual Testing.

Initial Investment in an Automation Platform is required, but it gets set off in the long run, especially when you consider the time saved, and faster go to market.

7

Cost of Manpower

Higher for Manual Testing as manual processes depend heavily on Software Engineers.

Much lower as automation tools automate and speed up the process, considerably reducing man-hours.

8

Skill Set

Experienced Software Engineers are required for Manual Testing. Limited level of programming knowledge and coding skills are mandatory for Manual Testers.

Programming knowledge is required for Automated Testing at the intermediate level only. However, with newer Automated Testing Platforms, the need to use coding or programming is also eliminated sometimes.

9

Risk Mitigation

Low-risk mitigation is subject to human error, fatigue, and the attention drop that comes with repetitive work.

Considerably high-risk mitigation as automated scripts once approved are not subject to future human fallibility.

10

Testing Productivity

Lower test productivity as manual testing processes can’t be recorded and hence can’t be reused. Furthermore it takes much longer to write individual test cases and test reports.

Much higher test productivity under Automated Testing, as the testing process can be recorded and reused to execute the same kind of testing operations in future too.

11

Test Coverage

In Manual Testing, test coverage depends on the skill and experience of the tester and is also subject to testers’ fatigue, memory, circumstances etc.

In Automated Testing, the quality of test coverage is much better throughout the Software Development Life Cycle (SDLC). Automation ensures that even the smallest unit is tested.

12

Geographical Access

Manual Testing offers only limited on-site access.

Wider geographical access is possible with Automated Testing, as access can be either on cloud or on direct device.

13

Random Testing & UI Changes

Manual Testing supports Exploratory, Usability and Ad-hoc Testing and is also useful where there are small changes in User Interface (UI) or where the Application Under Testing (AUT) changes frequently.

Automated Testing does not support random and ad-hoc testing. Furthermore, even small, cosmetic changes in the UI require automated test scripts to be changed.

14

Agile and DevOps

Manual Testing is not very conducive for Agile and DevOps, as Agile follows a test first approach and DevOps requires speed in the development and testing process.

Automation with its Build Verification Testing fully supports Agile and DevOps. Automated Testing is vital for Agile and DevOps as Agile is a continuous incremental model of testing and DevOps brings cross-functional teams together right from the start of the SDLC.

15

Regression Testing

Manual Regression Testing has its place especially for quality regression and for testing unusual cases in the production environment.

Automation is indispensable in Regression Testing, as it greatly reduces the time and cost of projects, by fixing defects early in the SDLC. Having a comprehensive suite of unit level regression tests, provides a safety net and makes sure that code changes do not break existing functionality.

16

Regression Volumes

Manual Testing is okay for low regression volumes but not cost-effective for high volumes.

Automated Testing is good for high regression volumes but not cost- effective for low ones.

17

Frequency of Functional Tests

Manual Testing is suitable if test cases have to run just a couple of times. It is not time-efficient for repetitive functional tests.

Automated Testing is very suitable when the frequency of executing the same test cases is high. It proves very time-effective in case of repetitive functional tests.

18

Performance Testing

Performance Testing has its limitations under Manual Testing, as it requires software performance to be tested at a given work load and at various given points in time.

Performance tests which include Load Testing, Stress Testing, Spike Testing, etc. are greatly enhanced with the use of Automated Tools.

19

Execution Across Operating Platforms

Parallel execution across operating platforms, comes at an additional cost under Manual Testing, since expensive human resources need to be increased.

In Automated Testing, parallel execution across platforms can be efficiently done, reducing test execution time, without a disproportionate increase in costs.

20

Human Touch

Manual Testing can give the app a human touch. It can reap the benefits of detailed human observation, intuition and insights and makes exploratory testing possible. It can greatly help enhance the visual aspects of UI like fonts, colors, sizes, contrasts, etc. 

Automated Testing which runs on algorithms does not involve human sensitivities and hence may lose out on personal touch benefits like user-friendliness and superior customer experience.

21

Documentation Quality and Accessibility

In Manual Testing, reports are recorded in Excel or Word and are not readily available at will for others. This also becomes a problem if another tester needs to take over for any reason.

In Automated Testing, test cases and reports are readily accessible by all stakeholders via secured login access. Additionally, documentation that gets generated has good training value, which helps new developers and testers to take over easily, especially in unit test cases.

22

Return on Investment (ROI)

ROI – an important consideration in any organization is comparatively lower under Manual Testing in the long run. This is more so in today’s fast changing digital world where speed decides who gets the first mover advantage. With the speed of technological innovation, comes a dire need for rapid testing, as investments have to pay off before technology gets updated or outdated.

The long term ROI is higher under Automated Testing, because of the increased productivity, cost-efficiency, time-efficiency, risk mitigation, etc. The heavy initial investment, paves the way for immense long term benefits. The product is ready for use much earlier and has a longer run for yielding returns before the technology gets outdated.