Since the complexity of the IT infrastructure increases, due to the overall number of the integrated systems, the need for grows respectively. The number and the complexity of the required tests grow, which considerably slows down product release. On the other hand, almost all companies aim to reduce software release time. That’s why the extension of the software life cycle due to the increasing testing phase is perceived negatively. Sometimes, it can lead to partial or even full refusal from testing which, in its turn, leads to the significant decline in the quality of IT services .
Neglect of software testing causes the following consequences:
- Reduced product quality (a prime cause of all the other implications)
- Constant growth of indirect costs for quality assurance due to software defects that appear during the exploitation period
- Loss of customers who do not want to use low-quality products
It’s desirable to conduct software testing as early as possible, at the stage of requirements analysis. Now let’s define the optimal moment to start testing.
According to the of the IBM company, the cost of software bugs removal increases in course of time.
For example, one hour of developer’s work costs 25$ and defect removal process at the stage of architectural planning takes 2 hours. Therefore, the overall cost is 50$.
In case the defect was found at the second development stage, the cost of its removal would increased fivefold and would be equal to 250$. The cost of bug removal at the testing phase will increase tenfold (500$), at the stage of beta-testing it will increase 15 times (750$). After the release, the overall cost will increase 30 times and will be equal to 1500$. As you can see, the cost of removing the same defect can vary from 50$ to 1500$ according to the time of its detection.
The chart below shows how the defect removal depends from the product lifetime:
The X-axis that corresponds to the timeline has a couple of dots:
A: the bug has appeared when the code was committed.
B: it took less than a day to find the defect that did not appear before, but was reproduced during the sequence of certain steps. It’s very easy to fix such defects.
C: a few days have passed. We do remember what changes were made to the system. And we know that these changes can be a possible cause of detected defects. This situation leads to the necessity to re-check the changes that have been made to the system additionally after the changes that caused the defect. Nevertheless, it’s not so hard to fix the issues without significant efforts
D: release date. Defect affects the customer and harms the integrity of your system. It will require some extra efforts to fix the defect in this case.
E: the defect in working software wasn’t detected for some time and have affected the parts of code which where not changed. Users know that it exists and have to accept its presence. It becomes harder to fix it since it penetrated deep into the system and developers have practically forgotten what changes caused this defect.
F: the defect in the system exists for a long time and the developer who made thechanges that have caused the defect doesn’t work in the company anymore. It’s extremely hard to fix such defect.
The green line shows the cost of all fixes during the time and consists of:
- Fixing the defect at the stage of development. At this point, the cost of the defect equals zero, since during the programming process developer can notice the defect and fix it immediately.
- Process switching – the exclusion of all the tasks besides the concentration on better understanding of functional and technical sides of the defect. If the defect was found by a developer right after the development process and before the testing phase, the cost of fixing the defect is near zero since the developer can quickly switch to it and fix it. In case, if other employee fixes the defect or we don’t know about its causes (just like in the F paragraph), the fix cost will increase many times.
- Determining the source of the defect. In this case, the cost increases gradually following the growth of the product complexity. The other significant factor is the fact that in the process of time people who knew what are the causes of the defect can forget about the changes that led to it.
- Fixing the defect after the release. The more functionality was added to the project with the unfixed defect, the harder and more expensive it will be to fix it. It also significantly increases the risk of new defects occurrence.
- The influence on the customer and your company’s reputation. The additional losses of funds may be caused by the client’s support and loss of the product’s image. The longer the defect exists, the more losses it causes..
The most famous example is losses of the Toyota company which were caused by the defect that was found after the product release. In 2009, Toyota recalled the sold cars due to sticking accelerator pedal. One of the passengers of Lexus ES350 has called the emergency service because the car started to accelerate uncontrollably at the speed of 100 km/h and didn’t respond to the brake pedal. Four passengers died. In November 2009 the dealers were instructed to shorten the accelerator pedal, update the software, and test the application with the defect which caused a delay in the braking system. By 2010, 8.2 millions of cars were recalled. It’s easy to imagine the overall cost scale.