Test Automation Done Right: Best Practices and Mistakes to Avoid

Having worked in test automation across various fields, I’ve seen both the good and bad sides of automation practices and I’ve learned a lot from my own mistakes. I want to share those mistakes with you so you can avoid them.

When building test automation frameworks, it’s easy to get caught up in coding and feel the pull to showcase your skills. However, in test automation, simplicity, maintainability, and usability are essential for long-term success. Below are some key mistakes to avoid, along with best practices to help you build a reliable framework.

1. When Complexity Kills Efficiency

I won’t dive too deep into every cause of overcomplicating architectures in test automation frameworks, but two main ones stand out based on my experience. First, it can be tempting to write complex, sophisticated code to demonstrate expertise. The second common cause is the misconception that test automation is a simple, repetitive task, leading some to underestimate the research and architectural design work required before building the framework. These two opposing approaches can ironically produce the same outcome: overcomplicated, hard-to-maintain automation code. In test automation, less is more.

When I say “less is more,” I don’t mean the code has to be overly simplistic. You can still tackle complex and challenging problems, just ensure that your code remains readable, maintainable, and scalable by following SOLID principles. Focus on the needs of test automation first.

Here are some examples that illustrate overly complex automation code:

• Keep the code hierarchy shallow to make debugging and navigation manageable. A shallow hierarchy makes the code easier to follow and maintain.
• Use abstraction thoughtfully. While abstraction can be useful, focus on the specific needs of the product under test. Over-abstraction can complicate the code, so ensure each layer serves a clear purpose that aligns with testing goals.
• Avoid generating dynamic test parameters on the fly. Ensure that global parameters affecting your test run are trackable through your IDE’s search tools by having pre-defined variables for them, so you can easily trace their values when needed.
• Limit the number of places where environment variables can be set. For instance, if a parameter is pulled from environment variables, it could come from various sources, such as an .env file, a command-line argument, or a default configuration file. Having too many sources to override the same variable can create confusion and issues during test runs. Consolidating environment variable settings helps prevent conflicts and makes the test setup clearer and more manageable.

2. Unlocking the Full Potential of Your Test Framework

While most test frameworks offer similar features, many also provide unique capabilities that can streamline your automation. It’s essential to understand both the limitations and features of your test framework. Documentation and GitHub pages are great resources for gaining this knowledge. Following GitHub pages also helps you track issues, bugs, and updates as well.

Understanding the specific features of a framework can save time and reduce complexity. Here are a few examples to illustrate the importance of knowing your test framework well:

• Imagine you’re familiar with frameworks like JUnit or VSTest but start using pytest in a new project. While these frameworks offer similar features, they have key differences that can affect how tests run. For example, in pytest, each test case is a new instance of the test class, whereas JUnit and VSTest create a single instance of the test class and run all test methods within that instance. Knowing this distinction is critical; if you structure your tests as you would in JUnit or VSTest, you might get unexpected behavior in pytest.
• If you’re used to adding explicit waits in Selenium to avoid timing issues, you’ll find that with Cypress.io, there’s no need for explicit waits. Cypress automatically waits for commands and assertions to resolve before moving on, eliminating the need for explicit waits or sleep commands in tests.
• Many frameworks have traditional setup and teardown methods, but in pytest, the preferred approach is to use fixtures. Adapting to this approach may take time, but trying to replicate traditional methods instead of using fixtures can make your code unnecessarily complex.

3. Better Logs, Smoother Tests

Good logging and reporting are essential to fully benefit from your test results and quickly identify issues. Logs help determine if a failure is due to a bug or an issue with the test framework. If it’s a bug, detailed logs make it easier to narrow down the problem and provide clear information to developers. If it’s an issue with the test framework, logs can significantly reduce the time spent troubleshooting. This is especially important in end-to-end or integration testing, where multiple components may be involved and detailed logging can help isolate the issue. Here are a few key points to keep in mind about logging in test automation:

• Implement comprehensive logging that captures relevant data, including failures, exceptions, the module or function name where the failure occurred, expected vs. actual results, configurations, and test environment details. For web UI testing, include browser type and version. For mobile testing, include OS, version, device model, and app version.
• Avoid random print statements. Instead, use logging libraries or tools that are supported by your test framework.
• Ensure that logs are easy to read and understand.
• For UI tests, enhance your logs with screenshots or video recordings.
• For API testing, logging request and response data can be very useful.
• Tracking how long each test run takes can also help identify potential performance issues.

4. Centralize Your Assertions

One of my pet peeves in test automation is placing assertions in multiple places, such as in helper functions or page objects. Having assertions only in the test case is a best practice and there are multiple reasons for it. Here are a few:

• When assertions are located directly within test cases, it is immediately clear what each test case is verifying. This improves readability by making it easy for anyone reading the test to understand the expected outcome and behavior being tested. It also provides a clear structure, setup, action, and assertion, which enhances readability and consistency.
• When assertions are scattered across different parts of the code or placed in helper functions, it becomes challenging to identify the cause of a failure. Centralizing assertions within the test case makes it easier to trace failures.
• Keeping assertions solely in the test case also helps you see if the test is becoming overcrowded. If there are too many assertions, it might indicate that the test should be broken down into smaller, more focused tests. Additionally, if you’re following the Single Responsibility Principle in your tests, keeping assertions in the test case helps ensure that each test case is responsible for checking a specific outcome.
• Finally, when assertions are in the test case, maintaining the test becomes easier. When requirements change, you can simply update the assertions in the relevant test case, without having to search through other parts of the code.

5. Document for Success

I know that documentation work doesn’t always get the highest priority on task lists; however, documenting how to set up the testing environment, run tests, and handle limitations is crucial for spreading knowledge across the team.

In large teams especially, proper documentation fosters good communication, helps new team members onboard smoothly, and enables them to contribute without constant guidance. While it may seem time-consuming initially, it pays off in the long run.

In fact, documentation isn’t just for large teams, it’s essential for any team’s long-term success. Whenever I encounter a project where documentation has been neglected, the project is often bound to face challenges or even fail. Why? Because even developers can forget the code they wrote themselves. After all, we’re only human. So don’t trust your memory, trust your documentation!

The specifics of what you document may vary depending on the product, but here are some general items that are universally beneficial:

• How to set up the test environment, including required tools and programs
• Permissions and accounts needed, with instructions on where to request them
• Code repositories, with brief descriptions of each
• CI/CD pipelines, schedules, and related details
• Test plans, if available
• How to run tests
• Limitations

Make documentation a requirement in your merge requests or user stories. It’s not just a nice-to-have; it’s a dependency for success.