student

CSE565 Fall 2010 | Week 04:Software Unit Testing Frameworks

javiergs


CSE565: Software Validation, Verification and Testing
Department of Computer Science and Engineering Ira A. Fulton School of Engineering
Arizona State University

 

 
Abstract

The present document is a result of a web searching investigation about Unit Testing Frameworks. It provides an overview describing what is a unit-testing framework, how would a developer utilize a framework, what benefits does the framework provide, what costs are associated with using a framework, and an example of a framework and the specific capabilities it provides.

1. What is a Unit Testing Framework?

Lets build the concept explaining each of the related ideas [1]:

  • Unit is the given name for a piece of code; for example, it could be a method inside of a class.
  • Unit Testing is a process that proposes that each Unit on the source code get individually tested (independent from other units). The goal of Unit Testing is to isolate each Unit and show that each of them is working correctly. In order to do that each Unit developer must define Unit Test Casesfor that Unit.

Now with the given definitions, lets define what is a Unit Testing Framework.

  • Unit Testing Frameworks are tools and libraries that help to implement the Unit Testing process. There exists different Unit Testing Frameworks for different programming languages.

2. How to Utilize It?

As a developer using a Unit Testing Framework means:

a)   To install the framework (tools and libraries) that is going to be used.
b)   Coding Unit Test Cases.

3. Benefits

The benefits of using a Unit Test Framework are:

  • It provides a contract (the Test Cases) that each Unit should satisfy, which reduces uncertainty about the Unit behavior.
  • It helps to locate and fix bugs early (during the implementation phase).
  • It provides a way to document the system. A Unit Test Case is a good way to document critical characteristics of a Unit.
  • It helps to re-think about the system modularization. Due the fact that each Unit will be independently tested, it is a priority to perfectly identify the responsibilities of the Unit (implementation) and its connections (interfaces) with other Units.

4. Limitations

The limitations of using a Unit Testing Framework are:

  • Unit Testing, by definition, only test the functionality of the Units themselves, it does not identify integration errors or performance problems.
  • It only shows the presence of errors (those that do not satisfy a Test Case) but do not warranty the absence of errors. This is important to keep on mind.

5. Costs

The costs associated with using a Unit Testing Framework are:

  • Developers need to write more code (to create the Unit Test Cases).
  • Writing more code means more resources. At least at the beginning it is going to be clear that more time or more people it is going to be required.

6. A Unit Testing Framework

The best way to give an example of Unit Testing Frameworks is to reference xUnit. xUnit is the name of a set of frameworks based on SUnit, a framework developed by Ken Beck for Smalltalk, and ported to Java for Erik Gamma and Ken Beck with the name of JUnit. Also ported to .NET asNUnit and ported to several other programming languages after that. Lets talk in specific about JUnit.

6.1. JUnit Capabilities

The main capabilities of JUnit are [4]:

  • It becomes the de facto standard for testing in Java.
  • It is recognized because it is easy to use (one test for each method).
  • It requires a minimal setup into the program structure to call the methods being tested and to compare the results of the call with the expected answer.
  • JUnit is a test framework, which uses annotations to identify the test methods.
  • JUnit assumes that the set of tests can be performed in an arbitrary order. Therefore tests should not depend on other tests.

6.2. JUnit Getting Started

To create a Test Case in JUnit basically, the developer need to follow three steps: (1) Import the package org.junit.Assert.*; (2) create a method and annotate it with @org.JUnit.Test; (3) call a method provided by JUnit to check the expected result of the code execution, versus the actual result, by example, call assertEquals().

If two or more Test Cases operate on the same sets of objects, JUnit allow us to define Test Fixtures [2] (those are preconditions needed to run a test, by example, instantiate the sets of objects). In order to do that follow the next steps:  (1) Add a field for each object that is part of the fixture; (2) Annotate a method with @org.junit.Before and initialize the variables in that method; (3) Annotate a method with @org.junit.After to release any permanent resources you allocated in setup; (4) Follow the steps described before to create the Test Cases [2].

7. Coding Test Cases

I will use JUnit to demonstrate the use of Unit Test Frameworks. In particular I will use Netbeans 6.9 and JUnit 4.5 [4].

7.1. Binary Search Algorithm

This algorithm requires that the collection of data in which we are going to search had been already sorted. Then the algorithm checks the element in the middle of the collection. If the searched element is smaller or greater than the middle element then a sub-array is defined which is then used to search again. If the searched element is smaller than the middle element then the sub-array is from the start of the array until the middle element. If the searched element is larger than the middle element then the sub-array is from the middle element until the end of the array. Once the searched element is found or the collection is empty then the searching is over.

Output

  • Returns the index of the element if founded or the negative index where the element should be inserted to maintain ascending ordering.

Input

  • The value to search.
  • The collection of values to search in. The input list must be sorted in ascending order prior to searching; if not the result of this method is undefined. It must also contain at least one element.

7.2. Java Code

This is a simple implementation of Binary Search in Java for integers. The class Search contains a public class method called contains that receive as parameters a sorted collection of values and the value to be searched.

7.3. Test Cases

It is not required to elaborate exhaustive Test Cases to show how JUnit works. So, just to give an example, consider the following Test Cases defined applying Equivalence Partitioning and Boundary Values.


7.4. Test Cases with JUnit

The class SearchTest implements the JUnit Test Cases as described in 6.2.

7.5. Results

Once the Test Cases are executed this is the output shown by Netbeans IDE.

Let us modify the method contains in our Search class to initialize the variable low as low = 1. While running the Test Cases we obtain an error message due the first element of the collection is never compared.

The error correspond to the line 34, where is defined the Test Case 1.

/*T1*/ assertEquals(Search.contains(a,  1),  0);

References

[1] Unit Testing. Retrieved September 5, 2010, from http://msdn.microsoft.com/en-us/library/aa292197(VS.71).aspx

[2] JUnit Cookbook by Kent Beck, Erich Gamma. Retrieved September 5, 2010, from JUnit web site: http://junit.sourceforge.net/doc/cookbook/cookbook.htm.

[3] JUnit Home Page, Retrieved September 5, 2010, from http://junit.sourceforge.net

[4] Writing JUnit Tests in NetBeans IDE. Retrieved September 6, 2010, from http://netbeans.org/kb/docs/java/junit-intro.html