Testing
Testing
All software systems need testing to determine if they perform
correctly according to their specifications. There are a wide range of
testing systems and several are covered here. Automating tests and
their reports is a critical part of software engineering. In some
systems, there is much more test code than the system under test. Perl
comes with several useful modules and CPAN has more all under the
Test:: namespace.
Unit Tests
This form of Testing focuses on the smallest Granularity of code,
typically individual subroutines and simple modules and classes. It is
also the best place to do coverage Testing as in most cases you can
create tests that exercise 100% of the target code. Perl comes with
several useful modules to make creating test scaffolds and reports. CPAN also has more test related modules.
Integration Tests
This form of Testing focuses on mid-level granularity of code,
typically subsystems and complex modules and classes. Regression
Testing is commonly done at this level. Coverage Testing can be done
but it is usually harder to get 100% coverage due to the increasing
complexity of the code under test.
System Tests
This form of Testing focuses on the highest level of Granularity of
code, i.e. complete running systems. These tests are usually conducted
in an environment (commonly called staging) as close as possible to
the real world. Because of the size of the code under test, Regression
Tests are most common here and code coverage testing is rarely
done. This is also a common granularity level to run load
testing and profiling.
Regression Tests
One common concern when developing large systems is making sure that
previously fixed bugs don't reappear (which is called regressing).
Checking for this is called regression testing. These tests are
usually run at the integration or system test level but even Unit
Tests can be called Regression Tests. Typically they are run with a
set of input data and a set of expected output data which are compared
to the actual output. The comparison can be very complex for large
systems and may have filters to ignore known or expected discrepencies
such as timestamps. As bugs are discovered and fixed, Regression Tests
should be added to make sure they don't resurface in later versions of
the code.
TLoad Testing
This form of Testing is designed to determine how a system behaves
under heavy use. It is usually part of itegration or system
testing. Input data can be randomly generated, played
back from logs, or manually created. Sometimes very subtle and
difficult to find bugs can only be found by long load test runs.
Coverage Tests
This form of Testing is usually done at the unit test level but
sometimes it is done at the integration level. It involves writing
tests that exercise each line of code under test. Creating Coverage
Tests is trickier than basic unit Testing as you need to create test
cases for all the possible code paths and sometimes for complex cases,
it is impossible. You also need special modules to instrument the code
under test that reports which code lines have and have not been
executed. CPAN has the Devel::Cover module which does this for Perl.
Profiling
If a subsystem has been determined to be too slow or inefficient for
the Requirements, then profiling can help determine the bottleneck.
Similarly to Coverage Tests, a special module is used to instrument
the code being profiled and it will report how much time is spent in
what sections of the code. Perl comes with the Devel::Dprog module and
there are domain specific profile modules on CPAN.
Benchmarks
Sometimes there are multiple ways to solve a software problem, be they
different code, algorithms or data structures. Benchmarking is a
special form of Testing which runs different code with the same input
data and compares their speed and/or resource usage (such as
memory). As with other tests, it is a skill unto itself to create
benchmarks that accurately isolate the problem and properly compare
apples to apples.
Copyright © 2005 Sysarch, Inc -- Last Modified Sat Jun 25 03:23:36 2005