Software development teams often embrace test automation with open arms, expecting it to be a silver bullet that finds bugs for them. However, reality sometimes ends up different. When requirements change all the time, and developers push new updates often, extensive test automation can become a maintenance burden rather than a quality safeguard.
Frequent Changes Make Automation Difficult
Consider a team that decides to automate everything possible. They write UI tests covering all user flows and API tests for most endpoints. It feels great at first — the team gains confidence that regressions will be caught before release.
But then the inevitable happens — as development continues, requirements change based on user feedback, technical constraints, and the positions of planets. Features are being added, modified, or dropped altogether. Suddenly, the automation starts breaking with each code change.
Now, instead of helping development, it’s become a ball and chain. Their “bug-finder” has transformed into an unstable beast that has to be constantly fed and groomed. Valuable development time is spent updating tests rather than building features.
The team faces a tough choice — pare down their automation to the bare essentials or become permanently bogged down maintaining it. Many teams find themselves in this situation when automation is not strategically implemented with long-term maintainability in mind.
Don’t Confuse Manual Testing with Automation
When starting your automation journey, it’s common to look at extensive manual test cases for inspiration and think that automating all existing manual tests is the best strategy.
But automating without considering cost, maintenance, and risk of change is a flawed path, leading to overly broad and unstable automation. Just because a test case can be automated doesn’t mean it should be. Manual testing allows for ongoing human observation, judgment, and efficiency. Automation expects rigid consistency and has zero adaptability.
Become more selective about manual cases to automate. Evaluate them based on a few factors:
- Stability — Will the test change often?
- Criticality — How important is the test case?
- Flakiness — Does the test depend on unpredictable factors?
- Cost — What is the relative maintenance effort?
- Coverage — Is the area already covered sufficiently?
Blanket automation of manual tests will result in an expensive suite requiring constant care. Your goal should have been optimizing testing speed and confidence, not maximizing automation coverage.
Each test case needs to earn its place in the code based on the criteria above. Otherwise, manual testing is often faster, cheaper, and more flexible. Automation is not a goal but a tool to provide smart coverage while speeding up releases.
Too Much Automation on UI Layer
Every product is a pyramid — interfaces at the top for users to interact with, supported by dense back-end logic forming the base.
It’s quite common to focus heavily on the tip of this pyramid when testing your applications, while the base with its API interfaces gets little attention.
A growing army of UI tests is exciting to build and watch running. Stakeholders and executives love them as they build confidence. The automated scripts walk through almost every application flow daily! Nothing can go wrong.
But UI tests are flaky and come with quite high maintenance costs. When the devs modify a page or workflow, dozens, sometimes hundreds of UI tests start turning red and blocking releases. It becomes an endless marathon of reruns and debugging while the whole team waits for the release to be rolled out. Imagine the stress.
UI tests are also slow. As more tests are being added, the test cycles start to drag from minutes to hours. After a while, you find yourself waiting all day for the suite to complete, only to find a few more failing tests still waiting to be fixed.
All that usually happens when the QA team has barely tapped into API-level testing. Those tests live closer to the code and don’t fail you when the UI changes.
Don’t put the cart before the horse, and start with the pyramid’s base. There’s a lot of low-hanging fruit before the heavy and ever-changing UI stuff. Unit and API tests will stabilize your foundation, while targeted UI tests can validate critical user journeys at the top. It’s all about balance.
Don’t Discount Exploratory Testing
In the rush to automate everything possible, it’s easy to overlook the value of exploratory manual testing. This hands-on, “thinking out loud” approach allows testers to interact with the system and make observations.
Exploratory testing uncovers usability issues and unexpected behavior of your product that will likely be missed by scripted tests. It provides an opportunity to try imperfect real-world usage scenarios and find interface issues that a robot wouldn’t notice.
Exploratory testing is manual and naturally inconsistent. Some teams rush to eliminate it in favor of “pure” automation. This is a mistake — skilled exploratory testers bring human perspective, creativity, and intuition.
Automation and manual testing complement each other. Keep exploratory testing part of your process, even if you decide to scale back its scope. Otherwise, you’ll have to compromise on quality and rely on users to report bugs and issues.
The most effective quality strategy combines automation for rapid running regressions along critical user flows with manual testing for in-depth evaluation. Please don’t fall into the trap of choosing between them. Use both approaches and determine where each adds value in your particular case. With the right balance, automation enhances rather than replaces manual testing.
The key is to carefully consider what to automate and how. Automating unstable parts of the app will require excessive maintenance. Aim for maximum reusability and abstraction. Prioritize critical path UI tests over extensive unit tests. Automation is powerful when implemented thoughtfully but can become an unmanageable burden otherwise.
Originally published on Medium.com