Virto Commerce QA Practices Supporting CI/CD Development of eCommerce Platform

This post is about QA processes in Virto Commerce as an ecommerce platform vendor. The goal of Virto’s QA team is to ensure the stability of the software environment on our partners’ side, help them to customize a Virto Commerce solution, and deliver it to customers. In essence, this is what we do in terms of QA to help our partners properly deliver Virto platform solutions to their end customers.

Another reason to write this post is to help our partners understand which parts of our product are tested - and how. We supply our software without bugs and with the best possible quality. As a result, our partners do not need to double-check these software modules.

1. VIRTO COMMERCE RELEASE STRATEGY
2. VIRTO COMMERCE TEST SUITE
3. VIRTO'S CORE DEVOPS PRACTICE
4. TOOL FOR ACCUMULATING QA CASES
5. QA WORKFLOW ON USER STORIES
6. PERFORMANCE TESTS AND REGRESSIONS
7. QA SERVICES PROVIDED FOR VIRTO SUPPORT TEAM
8. TEST AUTOMATION
9. SUMMARY

Virto Commerce is always focused on innovation and the rapid development of new functionality. New features and enhancements are shipped almost daily. We call them Edge releases.

Virto also has stable releases. These are released once per quarter and contain a collection of modules that have been tested and are guaranteed for compatibility and quality. The QA team is busy delivering hotfixes, researching bugs, and so on until a stable release is ensured. We use continuous updates for the current version in use, so the solution on the partner or client-side can be seamlessly updated.

This way, partners can choose different strategies and get innovations faster during active development or get stable releases to update any of their projects that are already running.

Partners have access to databases of test scenarios and their results. This eliminates the need for partners to duplicate tests that have already been run, reducing their workload and increasing efficiency

In addition to functional tests, we also regularly perform load testing. We monitor the platform’s reaction to the new release and its effect on the processing load. If we see that performance has dropped, we keep this release for further development before it is published.

Continuous integration and continuous delivery (CI/CD) are both a method of development that focuses on delivering code changes more frequently and correctly. We built Virto Commerce CLI (command line interface) and Virto Cloud as a PaaS (platform as a service) to provide our partners with the highest quality software products and services. To that end, we have adopted the CI/CD approach to our software development process.

As part of our CI/CD practice, we automatically build, test, and deploy each code change to a staging environment. This allows us to quickly identify and fix any issues that may arise. We also run comprehensive tests on every code change before deployment to production. This ensures that our software products are of the highest quality and free of bugs or security vulnerabilities.

The capability to write, test, and deploy code in a repeatable manner is crucial for Virto’s DevOps team to improve quality. It's also a key component of Agile methodologies like Scrum and Kanban. CI/CD allows software developers to focus on fulfilling business needs while maintaining code quality and security by automating integration and delivery.

We use Testrail as a repository of test cases to ensure that all aspects of the platform are covered. Currently, we have more than 2,800 test cases in our repository. This covers both core and non-priority modules. We regularly revise our test cases to keep them up-to-date with changes in the platform. In addition, we add new client scenarios and reproduce bugs to ensure that everything works properly.

Our QA process follows the two-week sprint cycle widely adopted in IT development. This allows us to deliver quality assurance for each user story in a timely and efficient manner. Sprint planning ensures that all stakeholders are aware of the tasks to be undertaken and the time in which they need to be completed. During the sprint, we track our progress in QA tasks and report any issues that arise. At the end, we review our results and plan for the next sprint.

This process helps us to deliver efficient quality assurance for our clients. It also allows us to effectively track our progress and identify any areas that need improvement: We are continuously working to better our process so that we can provide the best possible service to our implementation partners and B2B end-user clients.

QA Workflow on User Stories in Testrail is as follows:

1. Actualization and updating test cases for priority modules
2. Adding new coverage for low-priority modules (customer review, sitemap, etc.)
3. Adding new test cases for story or update cases for bugs (if needed)

In addition, testing user stories early and often is a best practice that will help ensure quality software. When it comes to user story testing, the Virto test engineering team checks whether a user story is functioning as intended.

For any bugs discovered, fixes need to be released. To do this, we select the cases we need to fill in the test environment, then send the results to the developers to create the relevant fixes. After that, the fixes are tested in the form of smaller tests.

The next activity is our load test, which we conduct twice weekly. It includes delivering the software modules to the load environment and launching the appropriate scripts.

From time to time we can detect some problems not related to functionality but more like a structural point or lost settings. So, periodically it is necessary to monitor and recheck the platform load parameters. This ensures our customers and partners get a solid, quality product.

Load tests generally monitor the parameters and make sure nothing exceeds specific threshold values. This cycle is repeated over and over again to ensure robust stability. We also perform activities such as regression testing for delivering a stable release. This activity is not very frequent— perhaps once each quarter—but it is a very time-consuming process.

The regression testing includes almost all cases and user stories from the previous quarter or even the past six months. Regression testing includes the following activities:

• Collection of modules for regression
• Preparation of regression sprint (delivery, create tasks, create test plan, other)
• Update Testing
• Create bugs
• Prepare internal report

The QA team often receives requests from support engineers or the Virto Professional Services team. In this case, the interaction algorithm includes the following steps:

1. Communication with QA from support or Professional Services
2. Help resolve the problem
3. Fulfill the bug or feature QA report

Time is also allocated for these tasks if help is needed from the QA team including recreating situations that produce bugs in our test environment.

The team has a large collection of automated queries (more than 1000) in Postman. On request, we share this collection with team members and partners. Postman is an API tool that allows the tester to send requests to services and work with their responses. It can be used to test the backend to ensure it’s working correctly.

The Virto QA team uses Postman for the following reasons:

• It’s intuitive and easy to use - and doesn't require any complicated setup or knowledge of programming languages. 
• It supports different APIs (REST, SOAP, GraphQL).
• It is extensible for all needs through Postman’s API.
• It easily integrates with CI/CDs.
• It runs on any operating system.
• It supports manual and automated testing
• It has a large community that can find answers to any questions

This is how we deliver releases, hotfixes, bug fixes, etc., to our partners and end clients. Our rigorous QA process ensures that partners do not need to double-check them, and are able to trust our quality. This allows them to focus only on the custom functionality unique to their business, scripts, etc.

Please contact me if you have any questions about CI/CD development and QA practices in Virto Commerce.