We Test Software, But How?
Hello everyone again. I will take a short break from automation and talk about what we should pay attention to in the software testing process. In this article, we will talk about what is software testing, what we should pay attention to, what important points are.
If our teas are ready, let’s get started. :)
What is Software Testing?
Software Testing is a process carried out to check whether the software product under test meets the desired requirements, to measure whether the product is at the expected level and to ensure that it functions properly as required. In this process, the product aims to eliminate all possible errors and to work in accordance with the requirements until it is presented to the end user. But let’s not forget that no product works 100% error-free, even a situation that may occur one day may be waiting insidiously :)
Test Principles
Test principles generally address issues such as what approach we should take and what we should pay attention to in the process of testing our product. It may not be possible to fully adhere to these principles in the process, but we can say that the more we adhere to the principles, the more useful and error-free the product we are testing will be for the user.
1- Testing Shows Presence of Defects
The test shows us the presence of errors. In other words, although our test is to improve the product, it should reveal the existence of errors at the same rate. The fact that we cannot find any errors at this point does not necessarily indicate that the software is error-free.
Tests should always be designed to find more bugs. The increase in the number of errors during the development process and the recognition of these errors is much better than the fact that the product is defective when it reaches the end user.
We must not forget that the error may be waiting for a special case to appear. We can say that this proves that no software product is 100% error-free.
2-Exhaustive Testing is Impossible
It is impossible to fully test the software product at 100%. In other words, it is impossible to evaluate all possible combinations and to include them in the testing process, both in terms of cost and time. I would like to give an example of this;
Let’s have a web page where we enter the ages of our employees. Let’s assume that the age ranges are between 22–50 and take an integer value. It gives values from -32,767 to 32,767 and 65535 probabilities. If we add special characters, invalid characters and other possibilities ( empty value, spaces) to this, it makes about 70,000 possibilities. If we were to spend a minimum of 1 minute for each eventuality, it would be 70,000 minutes, which would take up to 49 days.
Therefore, we must determine our risks and priorities during the testing process, and consider these risks and requirements while creating our scenarios.
3-Early Testing
Until recent years, testing was perceived as an issue that was addressed either at the end of software development processes or when the product reached the end user.
Testing is a process that should be started at the very beginning of the software development process. Catching potential errors at the requirement stage saves us a great deal of cost. Our goal here is to prevent the error from going to the next process. The more advanced the error is caught, the more cost and effort it causes.
4-Defect Clustering
The errors we encounter are usually concentrated and clustered in a particular area of the software. This approach also argues that the errors that occur are clustered in a certain area. Let’s say we have a software consisting of 10 modules in total and we have found 100 bugs at the end of the first sprint. Is it 10 errors per module or are they clustered at specific points?
According to the Pareto(80/20) Principle, for most events, roughly 80% of the effects are due to 20% of the factors. This shows up in many situations. Based on this, we argue that most of the errors are caused by a certain cluster.
5-Pesticide Paradox
How likely or reasonable is it to run the same tests over and over and expect different errors?
At this point, we should regularly review and refresh our test cases. If necessary, we should wait for different bugs to appear by adding new test cases on a certain sprint. In this way, we can recognize our potential mistakes early and offer solutions to eliminate them.
6-Testing is Context Dependent
We cannot test every software product in the same way. For example, if we test our tests and test scenarios that we run on an e-commerce site in a banking system or hospital software, the results we will encounter will not be the same.
So what should we do?
Our risks and requirements tell us which test type to use. We have to adjust our scenarios according to these risks and requirements. If there are scenarios and possible results that we have developed on an e-commerce site before, we can discuss our experiences and similar scenarios on an e-commerce site. Of course, at this point, risks and requirements should be at the forefront.
7-Absence of Error Fallacy
One of the biggest mistakes made is to argue that the product is completely error-free. As I mentioned above, a product can never be 100% error-free. We found our defects on the software, explained the situation to the development team, registered as a bug, bug fixes were made and tested again. The fact that the defects we find have passed the test does not mean that we have completely defected the product.
We should not forget that our goal is to find every possible error, perform the tests and sleep happily. But unfortunately that will never be possible.
I wanted to touch on the general operation of our tests with a short article. I hope it was a useful writing. See you in other articles.
