Everyone in the Senior Management echelon with the client's organization are convinced that test automation is the preferred way ahead to shrink the test life cycle and that test automation will certainly yield results not only in terms of saving the test execution time, but also considerable effort and cost can be saved through this. But many a times, test automation does not yield the expected results and on an ongoing basis it fails to deliver the cost and effort savings for all the organizations, business units or applications which are automated.
Many of the organizations that we are working with have already decided on a test automation tool or languages that can be used for testing across all their business units or applications. In some cases, different business units of the same organization have their own test automation tools to automate the applications (or) same tool licenses have been procured individually by each business unit, etc. The testing tool vendors make tall claims about the features of the testing tool and create an impression that test automation is just subscribing to the testing tools and that the test automation process is easy enough for anyone to automate the applications. The tool may not be a best fit for all their applications and an automation architect has to deploy a combination of tools and languages for automating the applications with various technologies.
Secondly, the absence of a robust common test automation framework across the enterprise causes the test automation to fail in various cases. The relevant framework has to be identified and developed by the test automation architect, which is scalable for the size of the applications and makes the life of the test automation engineers easy for scripts creation and execution. Every other day, multiple users friendly, script less automation frameworks are being evolved depending upon the complexity of the business applications, technologies, etc. Unless the right automation framework is identified, maintenance of scripts will be a problem and automation is bound to fail, in the longer run.
Again, the identification of the right applications for test automation gives the desired benefits within the expected time frame. There are multiple factors which need to be considered like the application stability, number of releases per year, application life, availability of a regression test suite, usage of regression suite on various releases, criticality of the regression for the business, application technology, applications which can be easily automated in a short span of time, etc. I know many clients who have invested heavily on test automation where it takes years to realize the ROI.
Test automation will be successful only if the test automation scripts are executed by the functional testing team on an ongoing basis. Functional knowledge is critically important to evaluate the correctness of the automation scripts. There is no meaning in creating thousands of scripts which do not catch your critical bug in the system. Initially they may require training or hand-holding by the experts, but this proves to be the best way to succeed in test automation. Nowadays, test automation is the need of the hour for agile releases where automation of test scripts has to be performed within the short sprints/scrum cycles. A foolproof, easy-to-use and functional tester friendly automation framework is required to help the functional team in agile testing.
Finally, in many places test automation has failed due to the maintenance issues. I have seen a client where the test automation scripts are always three releases behind which makes the test automation suite null and void. The maintenance model for regression should be robust and scripts should be updated immediately after the release, if not for the same release.
