The challenge
- Average defect lifetime in the project > 1 month
- A lot of manual testing before each release Integration between the main system and third-parties systems causes a lot of defects.
- Test environments are configured manually and the release takes several days to install
- Poor test data in the development testing level.
The solution
- Average Defect Lifetime decreased to 2-3 days
- Test expertise sharing between developers, components testers, and system testers led to a synergistic effect in the teams
- The use of more complete and complex test data led to the finding of errors in the early stages (defect costs decreased)
- Independent teamwork with the necessary continuous integration significantly accelerated the development and features release.
- The quality of the release has been greatly improved. Some of the quality metrics improved 10 times.
Project background
Customer: The leading Nordic digital services company with 24,000 experts globally and customers in more than 90 countries
Project description: The development project is the core of the banking system, which includes more than 20 subsystems. Development is carried out by 12 development teams. The project is divided geographically between 3 locations. Testing in the project was divided into levels:
- Testing in development teams
- System testing
- Non-functional testing
- User Acceptance Testing
Project size: big ( >450 employees)
Project duration: 8 years
Project implementation: 6 months
The Project Stages
01.
Duration: 2 months
The following basic concepts have been developed: Base Test Architecture Vision, Target Test Architecture Vision, and Transformation Strategy. Based on the Transformation Strategy, a list of technical tasks necessary to complete the Transformation Strategy was developed and a detailed transformation plan was developed. Metrics were approved by which it was possible to evaluate the progress and results of the transformation.
02.
Duration: 3 months
2 of the 12 teams were invited to participate in the Pilot Project. Teams were located in various locations. A plan for introducing a new way of working was developed for each team. Twice a week, progress was assessed and technical issues were discussed. Teams shared their solutions. The pilot project took 3 months and the successful result was presented to the rest of the teams.
03.
Duration: 1 month
In other teams, the transformation was postponed due to a large restructuring of the business. Those teams that participated in the Pilot Project continued to use the new way of working. Several more teams joined them on their own initiative and implemented the transformation over the next month.
Conclusion
- Test scope has been redistributed between test levels: Unit, Component, System,
- System integration, Acceptance. Most test cases have been moved to low-level stages (“Shift tests left”).
- All needed mocks for the Component test level have been developed
- Test data has been shared from the System test level for use in low-level testing (“Shift tests right”).
- System testers joined development teams and helped with the Component level tests development.
- Each team got its own feature environment.
- CI/CD process has been completed. Automated deployment to the integration environment and System test suites have been added.
- Continuous testing has been implemented.
- All needed documentation about the New Way of Working and new Test Strategy have been completed.
