The usefulness of unit tests based on a practical example

Unit tests are far less useful than repeatedly claimed. There is additional effort for the implementation and maintenance of unit tests, and the infrastructure has to be created first. Besides, as a good developer I know where the error occurs anyway. With good debugging skills I can find and fix every error immediately. Here is a small evaluation from practice.

Framework

The following is an architecture of a medium sized application. The user interface – corresponding to a web application – is mapped via HTML and JavaScript. The data is retrieved via services. These in turn query a business logic layer. The encapsulation of the different data sources takes place via repositories. The connected data sources are of different nature (DBMS, Sharepoint, etc.).

The testing infrastructure

A common excuse is the creation of a suitable testing infrastructure. Often, however, the wrong approach is taken here:

The testing infrastructure is not considered at the beginning of the project when the solution is initially set up. Only at a later point in time (often after numerous man weeks of implementation the need is recognized). Questions regarding the follow-up of tests, possibly necessary refactorings, in order to enable testing at all, drive the effort up. Unit tests are not introduced, or only half-heartedly.
Unit tests are set up as integration tests. Services often require configuration (connection settings, configured providers, etc.). Often this is done via configuration files in order to be flexible on the target system. Due to a lack of know-how regarding mocking or simple non-use, complex structures are created that have nothing to do with unit tests and considerably reduce the motivation to write tests due to the effort involved.

The basis for unit tests must therefore be considered at the beginning of a project. With a later extension refactorings must be accepted if necessary, likewise it applies to consider, which ranges are to be tested. A restriction to this as well as a strict adherence to it can improve the situation considerably.

What do I need for my testing infrastructure?

Regardless of all the discussion and over-the-top solutions available on the web, as a developer I need little in reality to successfully run unit testing:

Visual Studio already provides test projects that will satisfy most needs. Of course there are numerous frameworks that can be used here, but this does not have to be the case. This results in de facto no effort. The will as well as a basic knowledge regarding testing must be available.
Services, providers and co. mostly refer to integrated resources that require a configured system. Unit tests do not require the testing of these resources, but only of the functionality that has to be passed through for the call (or the business logic). It therefore makes sense to mock services. Numerous mocking frameworks (moq, Rhino Mocks etc.) are available for this purpose. There is little difference in the way they work. An initial training phase can be completed in between 30 and 120 minutes, depending on the know-how.

Useful base classes. Certain mocks or mocking configurations are used for most tests. Well-considered base classes facilitate the daily writing of unit tests immensely. As a rule, however, there is usually ongoing effort here due to the extension of the infrastructure. The initial effort is usually limited to identifying the initial requirements for the tests to be written.

As a rule, it is recommended to start with unit tests and to extend them successively. If the tests are taken seriously, this will have an effect on the entire infrastructure, which will then expand virtually by itself.

The biggest hurdle with unit tests is the inner pig. Once this is overcome, the unit tests almost write themselves. The second hurdle is the question of the parts of the software to be tested. It makes little sense to want to test everything. Rather, one should concentrate on essential areas of the software and test well those parts that make up the basis of the application.

Visual Studio already offers a .NET developer many facilitations. Together with a mocking framework and a portion of will, good results can already be achieved – and without much effort.

Evaluation from practice

The effort for the infrastructure must be accepted as a given. If necessary, this can be kept to a minimum, which is also expedient. It is rather the ongoing effort for the creation and maintenance of unit tests that counts. This is usually difficult to quantify. However, the following is an overview carried out in practice, which certainly does not provide an answer to all projects and framework conditions, but can be seen as a good point of reference.

It was found that approximately 60 bugs are found per QA run. These consist of UI bugs (different browsers, etc.), business logic bugs, and various other bugs in processing with third-party systems, etc. Depending on the area, these may or may not be detectable by unit tests (testing of JavaScript, etc. was not considered here) (so they have to be found manually anyway). Below is the corresponding list.

List of found errors

In this case, no unit tests were available for the affected locations. So the found errors had to be checked, found and fixed manually. In the course of this (the effort was recorded separately) unit tests were implemented for these errors (see detectable by unit tests). This usually took 10 minutes. Since the tests are sensibly written during the implementation and here of course the cases to be tested must first be found (or tests for the fair as well as the foul weather case must be implemented), 4 times the required time was estimated. The result can be seen in the following overview.

Comparison of effort for bugfixing and unit testing

The result shows a difference of 5.7 man-days in favor of the unit tests. All other cases cannot be detected by unit tests (some of them already operating at higher effort) and were not taken into account.

The numbers were determined in the course of a sprint of 4 weeks and a team of 3 persons. An extrapolation to a development year thus results in a difference of almost 74 man-days, which can be used elsewhere, assuming that the composition behaves similarly (which, however, will not be the case in practice). Nevertheless, this results in a clear advantage for the use of unit tests, since effort is definitely saved here – contrary to opinions elsewhere.

Conclusion

This calculation is based only on the experience of a single sprint of a development team and probably requires the collection of further sprints to refine the result. However, it may be assumed that the effort savings increase with increasing project size. Even in this example, almost 6 man-days could have been used more purposefully. Instead of troubleshooting, further functionalities could have been developed, documentation could have been written, or requirements could have been scrutinized more closely. For the sake of completeness, it must be mentioned that the QA team also needs time to record errors, the steps to get there, etc. This time was not taken into account … This time was not taken into account …

At this point I would be interested in your experiences regarding motivation, efforts and handling of unit tests in practice. How does it work in your development team?

Published
Categorized as News