7 Questions Developers Ask About Test Automation

4 min readApr 16, 2021

Software must be thoroughly tested on an ongoing basis both in terms of new features and assuring that the existing functionality was not damaged. While we know that test automation frees up more time to focus on delivering the level of quality that our customers deserve, I have noticed that we often don’t focus on the internal steps needed to deliver to our internal software development teams. While helping our internal dev teams adopt new test automation practices, I decided to compile a list of questions and responses that I have faced internally in scaling test automation adoption.

Q1. What are some common pain points different companies face while implementing automation?

Establishing automation takes time. Besides taking care of tests, automation development requires taking care of automation infrastructure, and there is often no available resources for such investment.
Reports of many automation frameworks are quite difficult to understand. If only the person who wrote the test is able to understand the report upon test failure, it naturally creates a bottleneck in releasing clean software fast.

We will see later on how choosing a proper automation tool can address these pain points.

Q2. What feedback should automation provide to facilitate fast detection and recovery upon regressions?

Make sure that your tests are reliable and do not report false alarms. Tests should fail if application under test behaves unexpectedly, and they should pass otherwise: make sure your tests do not fail because of bugs in tests, outdated expected results or flakiness issues.
Automation feedback should be relevant and only include actionable items. Remove tests (or create workarounds) to prevent reporting issues that are not planned to be addressed.
Having easy to understand reports will help to always find the root cause and solve the issue as soon as possible.
Automation should be stable. No matter how many tests you have, the above requirements should persist over time.
Lastly, the feedback should be helpful not only for testers. It should serve all involved stakeholders — product owners, project managers, team leaders, but most importantly — developers.

Q3. Why should developers be a part of test automation delivery?

Increase quality of your deliverables when you

Fail fast: detect regression upon code push rather than at testing stage, eliminating QA bottleneck.
Analyze fast: you know better than anyone else what was changed in the code that was pushed and what previously expected behaviour it could have impacted.
Recover fast: fixing a regression as earlier as possible decreases the cost of a fix.

Q4. Does it mean that we don’t need testers?

Nothing can replace a tester who has “fresh eyes” on the feature to explore it and make sure that it complies with product definition, eliminating requirements misunderstandings which can happen.
Besides, usually there are more developers than testers, and testing workload is high (especially considering repetitive regression testing of earlier releases which gets higher with each subsequent release). By assuring basic functionality behaviour you allow testers to concentrate on end-to-end testing which leads to more extensive software coverage.

Q5. But developers’ workload is high as well, and investing in testing can slow down development progress…

Undoubtedly! With that said:

Slowing down is not bad. Customers prefer smaller but properly tested releases. In the long term you slow down to go faster.
If you choose a proper automation tool, testing won’t take too much of your effort.
Following automation best practices will help you to scale up efficiently.

Key takeaways

Automation feedback should be reliable, relevant, stable and easy to understand by all involved stakeholders, but first and foremost it comes to help developers to fail fast -> analyze fast -> recover fast
Release less but better

Further mastering

Review the following posts to get a better understanding of the below topics:

Q6. How to choose a proper automation tool

Coded vs. code-less
Open source vs. commercial
Direct and indirect automation maintenance
Integration and Reporting

Q7. Automation best practices

Re-usability
Independence
Effort sharing
Achieving best potential coverage at lowest lost

About Katya

Katya is an Automation Developer at Cyolo with over 20 years of experience in business development in the areas of software development, implementation, application engineering, and automated testing. In her spare time, Katya enjoys photography.