Automated testing has become an essential part of modern software development processes.
To ensure the quality of their products, many companies rely on QA teams to create and manage automated tests.
In this article, we will take a closer look at how the Crisalix QA team handles automated tests using Ruby in combination with Capybara.
At Crisalix, our QA team has almost 500 automated tests, which are run on a dedicated test environment called Sandbox.
The Sandbox environment is separate from the production environment, which minimizes the risk of impacting the live application and allows teams to test freely and without constraint.
One of the main advantages of using a dedicated test environment like Sandbox is that it enables our QA team to experiment with different testing tools, frameworks, and methodologies without the risk of impacting the production environment.
All tests are run using GitHub Actions, an automated tool that allows the team to streamline their testing process.
The critical test suite is run every day using the Sandbox environment, which is the last stage before pushing to production.
It is critical to catch bugs before production deployment and prevent deploying any new version with critical or major bugs.
The all build test suite is run every Saturday using the Sandbox environment as well.
The goal is to catch all possible issues, even minor ones, and get fresh results before Monday morning so that the QA team can analyze and inform management about regression issues that could affect production but are not critical.
The regression test suite is triggered with Jira events.
The goal is to help the QA and DEV teams catch as many issues as possible during the testing and verification of new features or updates in existing features.
These tests are split into suites, so each team member can trigger them using Jira automation workflow for any Staging environment.
Triggering the regression test suite with Jira events ensures that tests are executed consistently and according to a trigger.
E.g We have a new feature and we want to check that this feature doesn’t break other functionality. The QA and DEV teams can select the tests suite in Jira ticket (for which this new feature is intended) and on GitHub automatically this tests suite starts in Regression Tests action.
By automating the testing process, Crisalix QA team can execute tests quickly and efficiently, saving time and reducing the risk of errors.
By linking Jira events to specific test cases, Crisalix QA team can track the status of tests and ensure that all required tests have been executed.
All of this is done by adding a Jira automation rule, where the team defines by which action they are sending a POST request to the GitHub side so it can trigger the execution.
As soon as the test suite runs finished, a comment appears in Jira ticket with a link that points to the result of the tests by clicking on which the team can see the results.
In conclusion, triggering the regression test suite with Jira events helps our team to automate their testing process, ensuring that tests are executed consistently and efficiently.
By considering factors such as test coverage, test data, test maintenance, and reporting, teams can ensure that the regression test suite is effective and relevant, supporting the delivery of high-quality software applications.
Despite the success of the automated testing process, there are always new challenges to face. One of the new challenges for the Crisalix QA team is setting up automation deploy on the Sandbox environment on a daily basis.
Additionally, the team is always looking to improve the performance of our tests and get feedback faster.
Finally, the team is working to help the QA and DEV teams in using automation tests for regression testing.