Allocate enough timing.

Working in the time box of a scrum sprint does present some challenges especially if your team is building a product from scratch. Sprint after sprint you have to add new features and ensure that whatever you built previously continues to work. Since in agile development there is a continuous release of various iterations, the team usually misses out on the quality. Even worse, when there are several sprints which require fixing, it becomes really tough for the team to devote time in developing, fixing and testing each iteration. It is necessary for the team to allocate enough time for testing. It is advisable to start test automation parallel to development so that there are no lags. This will help the team in becoming more productive and respond to the results quickly and avoid last minute rushes. Moreover, it will also help testers to devote time in exploratory test which requires manual efforts.

Test Automation Backlog

Maintain a test automation backlog for your project that contains all needed automation tasks and identified improvements. If you then target a few items from the backlog every sprint, in no time you will start to see the new regression test suite taking shape. Occasionally, stories from the test automation backlog may require dedicated developer time to implement and consequently some buy-in from the product owner in order to proceed. However, it should not be difficult to convince the product owner about the value of such stories if everyone on team is committed to quality. A test automation backlog could contain a prioritized list of items such as:

• Parameterize the test environment for test execution.
• Integrate with Continuous Integration.
• Enhance reporting mechanism.
• Provide an option to attach error logs in notification emails.
• Collect performance metrics for workflow scenarios.
• Add tests to check for concurrent execution of critical test cases.

Developing quality test scripts keeping in mind regression testing needs

Teams should develop quality test scripts keeping in mind the regression testing concept. Developing quality test scripts here means, ensuring that the test scripts meet the below mentioned criteria: correctness, maintainability, versioning, integrity, portability, performance. The objective behind this is to run testing especially regression testing smoothly and correctly without any intervention from the testers. If the test scripts are developed in the regression testing concept, the team would be able to complete the testing without many changes in the scripts and can also avail the benefits of performance, speed and accuracy.

Get the right automation tool

Testing team should understand the fact that if it's not the right automation tool, nothing's going to be right. A lot of effort should be involved while evaluating and purchasing a test automation tool. One should always refer to the criteria or requirements before selecting test automation. Below are some of the features/criteria that an ideal test automation tool should have/meet. It is also important to keep your tests and test data independent of the selected test automation tool as much as possible. Creating tests with hard-coded test data, system configuration, and properties makes them difficult to maintain. And in the long run, strong coupling with the test data makes it difficult to change your test tooling midway through the project if you were to run into any unanticipated issues.

• The test automation tool you choose should be OS friendly. Larger the number of operating systems it supports, better it becomes for teams to perform testing
• Test automation tool should also be in line with your budget and should incorporate features in terms of its pricing and licensing
• The tool should have easy to understand architecture. Meaning, the UI, navigation and functionality should be user-friendly so that even a layman can operate it. There are certain testing tools with codeless architecture which means, testers need not write codes and can test the applications in minutes
• The automation tool you choose should be able to give quick results and reporting abilities so that testers can quickly respond to the requirements of the agile testing environment

Keep a bird's eye on development environment.

The teams should understand and check their development environment which consists of simple virtual machines to cloud environments that deals with external services and database. Any bug or problem in the application may be the result of network problems, configurations, unavailability of services, backend data, etc. Without good understanding or monitoring of the actual environment, teams would be juggling with the root causes and would waste their time instead of achieving the results. Hence it is imperative to get complete knowledge about the development environment and their aspects before implementing test automation so that teams can divert their attention in improving the quality and not in finding the causes of defects.

Keep tests small and lean

Keeping tests cases small and lean really helps avoid the unnecessary test data that do not require testing or add no value to the actual result. It also helps in dealing with complex scenarios since the test cases are small. Moreover, teams do not require analyzing large suite of test cases which contain different codes, scenarios, configurations etc. Smaller and incremental test cases also reduces the load on the agile development environment and helps in evaluating the cost.

The lessons I learned while implementing test automation:

• Start small at the beginning of the project and build the test suite iteratively with each passing sprint.
• Create a test automation backlog to serve as a prioritized list of automation tasks. This will help you stay focused on the immediate tasks without losing sight of the long-term goals. Have a good look at the available testing tools and their capabilities and don't be afraid to invest a sprint or two in getting your hands dirty with them. This will ensure that you get started with the best available testing options and establish a realistic assessment of their features.
• Keeping your tests and data loosely bound will help you to switch testing tools with ease in the future if such a need arises.
• Create meaningful tests and give due consideration to concerns like maintainability and execution time while adding tests to the automation suite.
• As soon as possible, make every effort to enable the whole team to use the security net you've created by putting it on a build/CI system.
• Create meaningful tests and ensure that they don't create a false sense of security.
• Make every effort to resolve test failures quickly and keep test execution times as short as possible.
• Have an intuitive reporting mechanism in place and give everyone on the team visibility to test results and historical trends. This will help everyone involved in the project monitor the progress and health of development and make more informed decisions.