In this simple article, I want to share some thoughts about how to automate a manual regression test plan . I did a simple poll where I asked :
Which tool/framework do you use to automate the execution of a full manual regression test plan to validate business scenarios ?
Before answering, you should think about the “Ice-cream testing cone”
Ice-cream cone : If you are testing through the User Interface (UI) them it will be slow, increasing build times. For more information please click on this article by Martin Fowler.
Ads: curso práctico de testing automático presencial y también online
From the previous image, it is clear that it is not a good idea to invest much time building a huge collection of UI Tests and here we have the results of the Poll :
I don’t think using Selenium or other UI Test Frameworks to automate the manual regression test plan is a good idea. Let me explain why:
- Maintenance of Selenium tests is expensive because the tests will be broken very often.
- They are expensive to create.
- They are slower than unit tests.
- They don’t give metrics about performance.
- They are not designed to test different scenarios concurrently.
Rich JavaScript UI should have most of the UI behaviour tested with JavaScript unit tests using something like Jasmine.
Ads: curso práctico de testing automático presencial y también online
In my opinion, integration testing is the best option for automating manual regression testing.
There are alternatives to automate the manual regression test plan like Soap UI or Visual studio load test and here are possible benefits:
- Cheap to create tests.
- Easy to maintain.
- Focus on the tests if the back-end process of the feature is working fine.
- Add extra value to determine the performance.
- Created for managing concurrence of multiple user tests.
What do you think about this?
Here we have other interesting articles about this topic :
- Test pyramid by Martin Fowler#
- The Practical Test Pyramid by Ham Vocke
- Gaining confidence without integration tests by Alister Scott
- Ultimate Guide to Automation Testing By Joe Colantonio
Ads: curso práctico de testing automático presencial y también online
Please share with me your thoughts about this topic via twitter or add a comment on this article here below 😉
I know that depending on the scenario the test pyramid should be modified. For example, it makes more sense to have more integration tests than unit tests.
I’ll post another article for that scenario.
Ice-cream cone and assets designed by @freepik
Ads: curso práctico de testing automático presencial y también online