This post was written a few years ago. It may have outdated dates and content and may not reflect my most current opinions.

During about the 4 years I have on the Software QA world I’ve been researching about the new techniques, methodologies, etc., on the field. Since the beginning almost empirically I’ve found on different articles various techniques, methodologies, terms. But above all of the terms appears one: Automation.

It seems that the trends go there on the Software QA world, and that’s what we are going to be doing most. But let’s see first what is this all about.

What is Automation?

If we do a quick search on the internet, we can find numerous articles defining the term “Automation”. Automation is, more or less, the action to test software using another software source that is unrelated and independent from the software that is being tested.

Well, we can say automation is the most actual technique, the “newest” thing. But we have to ask, automation is what we need for this project?

Automation is for all the cases?

Do we need to automate? why automate? when automate? how much is the cost of automation?. On my very little experience on this field I’ve reached a conclusion: Automation is not for every case. This are some advantages and disadvantages about automation that you can take it to make the decision to use it or not.

Advantages

• Time saving on the future. Although it requires more time at the beginning, it saves more time in the future, when the tests are repetitive or recurrent tasks. When something is automated we can only start the software and perform the tests without any effort. Maybe some adjustments, if the Software under testing changed.

• Tediousness elimination. and the time invested on perform tests. Let’s say that we have to test a series of 100 different test cases. We can prepare the data and test every single test case using Automation. If we do it manually, depending on the tests, could take more much time and effort.

• Helps the Tester to avoid wasting time on countless tests. As on the previous point, being automatically done, the tester can dedicate to other tasks, while the tests are being perfomed and use the time to review results and check for other bugs on the software being subject to testing.

• Better sampling and randomness. We can do better and more tests. Investing less time on scripting a test that will run hundreds of tests, the Tester can run a lot more different tests than manually in less time. The Tester can prepare more randomized data in order to get better results and seek for more bugs. Giving more reliability to the tests performed. Is better to test something with 1000 users than 100, and better using 100 different photos with different sizes, formats, etc, than 10 of them.

Disadvantages

• It requires more time at the very beginning. It requires more time and effort to setup a framework. It can consume too much time at the beginning so that means start really testing can be later than if the tests were performed manually.

• Could require bigger investment. There’s free and commercial software out there, but we need to analyze our needs and resource to research which testing software is better suited for our project. The software that I’m testing is a desktop app or is it web?. Are I or my team quailified enough on this Testing Software? How much can I spend on training or software licenses? etc, just some examples of what we have to ask us first.

• Time investment on the long term. If the software under testing is changing quickly over time, it can be inconvenient because the Tester could end spending more time updating their scripts instead testing the application.

In conclusion, before deciding if we are going to automate we have take into account some of the following:
1. “The application that I’m going to test will be more or less static?”. Let’s automate!, on future tests you will just need to add new data and maybe some times just tweak your scripts.

1. “How many tests do I need to do? how many test cases per test? how many different users do I need to use?”. For example, if a field has 80 different possible options, definitely you have to automate to get all of them. If you need to run 3 tests, don’t waste your time, do them manually.

2. “We are going to work in an application that we are going to need review it on a daily basis”. Let’s write some scripts to invest our time on better things.

3. “My coworkers and I are working on a project that has to be done completly today”. What are you waiting for? start testing. Don’t waste time recording, editing, debugging, tests only to start testing after all that.

Finally, we have to consider all these points when your are deciding to automate or not and how much is going to be beneficial on the short and the long term. But definitely automation is a great tool when we learn when and how to use it.