Evolution of software testing techniques

Modern requirements to the reliability of information technologies

Information technologies play a more and more important role in the modern world. It is virtually impossible to imagine the absence of information technologies in various areas of life. Nowadays, information technologies in business no longer give an advantage over competitors, but are rather an attribute of competitiveness.

Information technologies cover modern business so much and get into it so deeply that their absence gives a company that ignores the world trend of computerization for some reason no chance to survive.

Together with the expansive growth, information technologies develop intensively as well. The complexity of software products increases together with the complexity of tasks they solve, requirements concerning CPU performance and network bandwidth also increase, which affects companies' expenses on the development and maintenance of their information sector. Companies' expenses on information technologies increase each year and this trend does not change.

Because of increasing expenses, requirements to the reliability of hardware and software cannot but get higher as well. But what increases requirements to reliability even more is how deeply information technologies get into companies' business processes.

Software product development process

Quality assurance tests play an important role in the process of creating software products. Measures ensuring that the quality of a software product meets the requirements are welcome in every step of creating the software product. Testing is one of the final and most important steps in creating a software product.

The Rational Unified Process (RUP) is an iterative software development process framework created by the Rational Software Corporation, since 2002 a division of IBM. The RUP is well known as the library of best practices of a project management.

The RUP "Test discipline" describes the software testing process. The purposes of "Test discipline" are:

• to verify the interaction between objects;
• to verify the proper integration of all components of the software;
• to verify that all requirements have been correctly implemented;
• to identify and ensure that defects are addressed prior to the deployment of the software.

Testing applications

The task of testing applications comes down to checking how changes in the source code of the software product influence its entire functionality and quality.
Testing can be divided into two types: full and partial.

Full software product testing

Full testing implies testing all functions and visual elements in a software product. The advantage of full testing is detecting all bugs and defects in the software product. The disadvantage of full testing is that it requires a considerable amount of time.

To reduce the time necessary to test large software products manually, developers use automatic tools for testing applications - unit tests- that are supported by the methodology of software product development called TDD (Test Driven Development). The main drawback of TDD is that unit tests are program code that has to be modified as well in case the source code of the software product is considerably modified. Besides, unit tests have a limited scope of use, in particular they are used only to test code libraries. Taking into account expenses on creating unit tests themselves, the efficiency of their use increases together with the increase in the stability of the software product, which, in its turn, reduces the motivation for their use. Another question left unsolved is testing unit tests themselves.

Partial software product testing

Partial software product testing means testing only those parts of the program code that have been affected by changes. At the same time, you should take into account both those parts of the software product that were actually modified and those parts of it that the modified parts indirectly affected. Only an expert that possesses the necessary knowledge about the software structure can detect such dependencies.

Partial testing allows you to considerably reduce the amount of work related to testing, including both testing and the preparation required for it, for instance, when unit tests are used.

Another means of reducing the time necessary to test the quality of a software product is statistical software code analyzers.

The reasons for their use come down to reducing the amount of code that requires to be analyzed by a human. Here it is also possible to use tools for target diagnostics of certain properties in the program under development. You can take the statistical analyzer Viva64 (http://www.Viva64.com) as an example. It is used to verify the code of a program when it is ported to the 64-bit platform.

Need for selective testing

Selective testing is especially relevant when you manually test the user interface of a program. Currently, there are no tools for automated user interface testing that are effective enough and that can compete with manual testing in terms of expenses and quality.

Testing optimization

The optimal process of testing a software product looks like this. First, you test the entire software product, all its functions and visual elements. As you make changes in the program code, full testing becomes unnecessary. You only have to test those parts of the program code that were affected by the changes.

This approach saves a lot of time, which allows you to increase the quality of testing.

Problems with using the optimal testing process

Despite the obvious advantages of selective testing, there are a number of practical problems that make this task difficult.

Absence of complete information about the product structure

The main problem that stands in the way of selective testing is the absence of reliable information about all modifications in the code of the software product and also information about the influence these modifications have on the rest of the software product. This issue is especially urgent when several developers develop one software product. Experience shows that all attempts to establish effective horizontal channels for knowledge exchange between the developers have no result due to the great complexity of software product code – every developer has more than enough information to deal with within his own area of responsibility.

Gap between developers and testers

It is even more complicated to give the necessary information about the internal structure of a software product to testers who, as a rule, have a lower qualification than software developers.

Features of the .NET technology supporting the optimal testing process

The task of automating the analysis of modifications in the code of a software product is effectively solved in software products implemented on the basis of the Microsoft .NET Framework platform (http://www.microsoft.com/net/). No matter in what language a software product is written, after compilation its code is saved in the intermediate language - MSIL (Microsoft Intermediate Language) and remains available for future analysis. Thus, no matter what programming language the team of developers use, the Microsoft .NET Framework platform makes it possible to use a unified way for analyzing the code of the software product.

Technique

The technique of analyzing changes in a software product makes it possible to single out the areas in the software product that have been both directly modified and indirectly affected by these changes. All these areas in the software product may contain errors, both evident ones made during the modification of the code and those present in other parts of the code that depend on the modified code.

To perform an analysis, you should build the correct graph of calls in the software product being tested and compare each method in the graph with its previous version in the software product that has already been tested. All changes found in methods should be moved up to the top of the call graph and thus you should detect all areas in the software product that depend on these changes.

Support of the technique by the Testing Relief software product

Currently, the only software product that supports the selective testing technique is Testing Relief (http://www.TestingRelief.net). Testing Relief does the expert's work - it analyzes all changes in a software product and detects parts of the program code that depend on these changes, but have not been modified themselves.

Thus, Testing Relief shows the whole picture of changes in the software product and determines the amount of work necessary to perform testing.

Exchanging knowledge about the software product architecture

Testing Relief allows an effective exchange of knowledge about the architecture of the software product between members of the team of developers. Exchanging information about links between methods makes it possible to detect all changes in the software product and their influence on the entire software product at the stage of preparing it for testing.

Logical links between methods that Testing Relief cannot detect itself due to its limitations are also important information. For example, event handlers should be always connected to the method that triggers this event. Let us take an example when an additional thread is created in an application. The following sample code shows how to create an additional thread.

private Thread _backgroundThread;
        
private void prepareThread()
{
   _backgroundThread = new Thread(new ThreadStart(_backgroundThread_DoWork));            
}

private void startThread()
{
   _backgroundThread.Start();
}
        
private void _backgroundThread_DoWork()
{       
   // do some work
}

As you can see from the example, the method «_backgroundThread_DoWork()» is called from the method «startThread()», despite the fact that the link to the method «_backgroundThread_DoWork()» was obtained in the method «prepareThread()». Thus, the method that calls the method «_backgroundThread_DoWork()» will always be the method which starts executing the new thread:

_backgroundThread.Start();

In case of events triggered by control elements on forms, the form constructor can be assigned as the calling method because a control element created on the form is supposed to be used.

Software developers and a quality assurance engineer participate in the process of preparing a software product for testing. Every developer specifies links between methods in the software product within his area of responsibility using either attributes or Testing Relief features. Also, any developer can create a list of links between methods in any form and the quality assurance engineer will have to enter this data into the Testing Relief project.

Outlook for testing technique development

High requirements to the quality of software products caused by the deep integration of information technologies into business life make us look for ways to increase the effectiveness of software testing.

Currently, the most flexible and effective method of testing user interfaces in software products is manual testing. The technique of selective testing supported by Testing Relief makes it possible to considerably reduce expenses on testing user interfaces and, at the same time, increase the quality of testing due to revealing all hidden areas in the code of software products affected by changes made in them.