Load and stress tests, automated regressive tests on mobile devices, and automatic generation of test data masses are just a few of the efforts that are increasing software quality standards for this international fashion chain store.
Fashion is on digital
The client is one of the world’s largest fashion chain stores and financial services networks. It is present in over 24 countries in Europe, Asia, and Latin America. In Brazil they have over 280 stores in 125 cities, and over 15,000 employees. In recent years they have been investing in digital technologies to provide an innovative experience for their customers. Digital is in fashion. Fashion is on digital.
Black Friday digital without falls and slowdowns
Special business dates are a challenge for IT. Marketing does its part by bringing thousands of consumers to stores and digital channels. At this point, it is important not to lose sales and provide their customers with a perfect shopping experience.
Distances and interruptions in the servers may cause the entire marketing effort to be lost. Such cannot happen! Not when the reputation of a great brand is at stake. The customer’s IT team was aware of how important was to correctly design its infrastructure for supporting Black Friday’s traffic peaks. After several attempts at other suppliers, Prime Control was consulted and it charmed everyone on the runway.
The challenge was to perform load and stress tests on a great diversity of the customer’s applications, systems and environments such as Middleware, OpenShift and MongoDB. The adopted tool was JMeter, a mature open-source tool, full of resources, very reliable for having a large community and, of course, at no licensing cost for the customer.
Results
Two test batteries were conducted. The first aimed at identifying application bottlenecks. Prime Control produced a detailed report on the identified problems and risks. Based on this report, the different development and infrastructure teams were called on to promote adjustments and improvements. Then, the second test battery validated the adjustments, indicating applications’ performance gain and better use of computational resources.
So the systems were optimized and could now handle a higher number of users without the risk of interruptions or slowdowns. Database searches were optimized, and a better balance of the server was also achieved.
Some flow targets for the systems were also established, and the expected number of users on business dates was simulated. An endpoint should get to four times the current flow as provided by the application. In a stress test, such endpoint would have to support the threefold of the requests expected for the first target.
For the customer, the need to perform continuous performance tests regardless of events such as Black Friday became clear. With this, demands for load and stress tests emerged for other applications in the customer’s technology base.
Automation is in Fashion
Tests are a critical mission for any development path. These are the tests that prevent bugs from being published, protect the user experience, the reputation of the brand, and everyone’s security. However, as this is one of the last stages, it is common for many companies to neglect such tests. But not for this customer!
The customer’s Quality Assurance (QA) team performed a given set of manual tests on their smartphone applications, on the iOS and Android platforms, which required approx. 3 days of intensive testing.
Prime Control was involved to build a reliable layer of automated regressive tests. This way, robots would perform tests, release the QA’s creative potential to perform tests or even prevent the appearance of bugs.
Using Python, Appium, and a Prime Control exclusive framework 25 robots were developed for regressive tests on several interdependent features of the customer’s Apps. In addition, 24 other robots were developed for “smoke” tests, half for iOS, and half for Android.
All tests focused on critical system functions that in the event of failure would seriously harm the user experience, such as user authentication, functionalities of shopping cart, product search, delivery address changes, among others.
Results
Over 5 days, manual and automated tests were working side by side. We were able to compare their performances. On the fifth day, the robots were performing 80 complete test batteries, while the manual test team had only carried out 5. Numbers do not lie! 8 working hours of the robot equal 128 hours of manual testing. A robots crushing win!
But don’t worry! All the QAs remain very busy, but now on more creative and preventive tasks. Test automation is a success.
Quality integrated to Jira
For control and audit purposes, it is essential for tests to produce evidence. It is through evidence that developers can investigate the root causes of bugs and produce important indicators to measure the quality of the software delivered.
However, the logs and screen captures at each step of the tests were done manually. This procedure was very exhausting and the cause of failures, lack of evidence, and a lot of rework.
The solution found by Prime Control was to develop an intelligent integration to Jira, an Atlassian software product that allows for an advanced management of bugs and incidents. Now, the QA team chooses what regressive tests to perform and the smart integration updates the status of these processes on Jira and attaches all logs and bugs evidence.
Integration to Jira generated a 110-hour savings in processes that were hitherto performed manually and took 120 hours. So the robot can perform all tests within 10 hours.
Four of the five people who previously performing the process manually were reallocated to other activities. Now, only one person is responsible for starting and monitoring the entire automated set of regressive tests, fully integrated to Jira.
The risks of lack of evidence were virtually eliminated since the process runs automatically. This means a much reliable audit since logs of evidence are adopted and incorporated to Jira at each execution.
Generating data mass in just 2 hours? Fantastic!
Masses of data, as the name itself suggest, are sets of fictitious or real data used to test the applications. They are used to fill fields simulating the action of a real user. A good mass of data is the first step for any reliable test.
The customer did not have a mature, documented process for generating masses for tests that involved billing for the logistics area. Knowledge was in people’s heads, but there was no pattern. This caused the centralization and funneling of requests, delays, and a certain dependence on specific professionals.
The deadline to generate a set of mass data was 5 working days on average. If the generation was done uninterruptedly, the customer estimated that 2 hours of continuous work for each item of this set would be necessary.
Prime Control’s solution started with the process documentation and produced a flowchart that took into consideration all its conditions. Then, the construction of a smart robot begun fully built on open-source technologies, license free, and easy to maintain, for data mass production.
The results impressed the customer. A single day of work of the mass generation robot equaled 14 weeks of a dedicated professional. It is an impressive gain of performance, trust, scalability, and standardization.
In a 2-hour race between a robot and a human being, the human would have delivered just 1 item for the mass set, while the robot would have delivered 70 ready-to-use items.
Process robotization drastically reduced the number of working hours a demand needs. The robot interacts perfectly with the other systems without the need or customizations.
The customer is very satisfied. He knows that being in fashion means to provide the customer with an incredible shopping experience, whether in a physical store, the app, or on the website. Prime Control is proud to have contributed to such success. Fashion is also tech!