Regression testing
Encyclopedia
Regression testing is any type of software testing
Software testing
Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software...

 that seeks to uncover new errors, or regressions
Software regression
A software regression is a software bug which makes a feature stop functioning as intended after a certain event...

, in existing functionality after changes have been made to a system, such as functional enhancements, patch
Patch (computing)
A patch is a piece of software designed to fix problems with, or update a computer program or its supporting data. This includes fixing security vulnerabilities and other bugs, and improving the usability or performance...

es or configuration changes.

The intent of regression testing is to ensure that a change, such as a bugfix, did not introduce new faults. "One of the main reasons for regression testing is that it's often extremely difficult for a programmer to figure out how a change in one part of the software will echo in other parts of the software."

Common methods of regression testing include rerunning previously run tests and checking whether program behavior has changed and whether previously fixed faults have re-emerged. Regression testing can be used to test a system efficiently by systematically selecting the appropriate minimum set of tests needed to adequately cover a particular change.

Background

Experience has shown that as software is fixed, emergence of new and/or reemergence of old faults is quite common. Sometimes reemergence occurs because a fix gets lost through poor revision control
Revision control
Revision control, also known as version control and source control , is the management of changes to documents, programs, and other information stored as computer files. It is most commonly used in software development, where a team of people may change the same files...

 practices (or simple human error in revision control). Often, a fix for a problem will be "fragile
Software brittleness
In computer programming and software engineering, the term software brittleness refers to the increased difficulty in fixing older software that may appear reliable, but fails badly when presented with unusual data or altered in a seemingly minor way...

" in that it fixes the problem in the narrow case where it was first observed but not in more general cases which may arise over the lifetime of the software. Frequently, a fix for a problem in one area inadvertently causes a software bug
Software bug
A software bug is the common term used to describe an error, flaw, mistake, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways. Most bugs arise from mistakes and errors made by people in either a program's...

 in another area. Finally, often when some feature is redesigned, some of the same mistakes that were made in the original implementation of the feature were made in the redesign.

Therefore, in most software development situations it is considered good coding practice
Best Coding Practices
Best coding practices for software development can be broken into many levels based on the coding language, the platform, the target environment and so forth...

 that when a bug is located and fixed, a test that exposes the bug is recorded and regularly retested after subsequent changes to the program. Although this may be done through manual testing
Manual testing
Manual testing is the process of manually testing software for defects. It requires a tester to play the role of an end user, and use most of all features of the application to ensure correct behavior...

 procedures using programming techniques, it is often done using automated testing tools. Such a test suite
Test suite
In software development, a test suite, less commonly known as a validation suite, is a collection of test cases that are intended to be used to test a software program to show that it has some specified set of behaviours. A test suite often contains detailed instructions or goals for each...

 contains software tools that allow the testing environment to execute all the regression test case
Test case
A test case in software engineering is a set of conditions or variables under which a tester will determine whether an application or software system is working correctly or not. The mechanism for determining whether a software program or system has passed or failed such a test is known as a test...

s automatically; some projects even set up automated systems to automatically re-run all regression tests at specified intervals and report any failures (which could imply a regression or an out-of-date test). Common strategies are to run such a system after every successful compile (for small projects), every night, or once a week. Those strategies can be automated by an external tool, such as BuildBot
BuildBot
BuildBot is a software development continuous integration tool which automates the compile/test cycle required to validate changes to the project code base...

, Tinderbox
Tinderbox (software)
Tinderbox is a software suite that provides continuous integration capability. Tinderbox allows developers to manage software builds and to correlate build failures on various platforms and configurations with particular code changes....

, Hudson
Hudson (software)
Hudson is a continuous integration tool written in Java, which runs in a servlet container, such as Apache Tomcat or the GlassFish application server. It supports SCM tools including CVS, Subversion, Git and Clearcase and can execute Apache Ant and Apache Maven based projects, as well as arbitrary...

 or Jenkins
Jenkins (software)
Jenkins, previously known as Hudson, is an open source continuous integration tool written in Java. The project renamed itself after a dispute with Oracle who claims the right to trademark the Hudson name and has applied for such a trademark as of December 2010...

.

Regression testing is an integral part of the extreme programming
Extreme Programming
Extreme programming is a software development methodology which is intended to improve software quality and responsiveness to changing customer requirements...

 software development method. In this method, design documents are replaced by extensive, repeatable, and automated testing of the entire software package throughout each stage of the software development cycle.

In the corporate world, regression testing has traditionally been performed by a software quality assurance
Software quality assurance
Software quality assurance consists of a means of monitoring the software engineering processes and methods used to ensure quality. The methods by which this is accomplished are many and varied, and may include ensuring conformance to one or more standards, such as ISO 9000 or a model such as...

 team after the development team has completed work. However, defects found at this stage are the most costly to fix. This problem is being addressed by the rise of unit testing. Although developers have always written test cases as part of the development cycle, these test cases have generally been either functional tests
Functional testing
Functional testing is a type of black box testing that bases its test cases on the specifications of the software component under test. Functions are tested by feeding them input and examining the output, and internal program structure is rarely considered .Functional testing differs from system...

 or unit tests that verify only intended outcomes. Developer testing compels a developer to focus on unit testing and to include both positive and negative test cases.

Uses

Regression testing can be used not only for testing the correctness of a program, but often also for tracking the quality of its output. For instance, in the design of a compiler
Compiler
A compiler is a computer program that transforms source code written in a programming language into another computer language...

, regression testing could track the code size, simulation time and time of the test suite cases.

Regression testing should be part of a test plan. Regression testing can be automated.
Regression tests can be broadly categorized as functional tests or unit tests. Functional tests exercise the complete program with various inputs. Unit tests exercise individual functions, subroutines, or object methods. Both functional testing tools and unit testing tools tend to be third party products that are not part of the compiler suite, and both tend to be automated. Functional tests may be a scripted series of program inputs, possibly even an automated mechanism for controlling mouse movements. Unit tests may be separate functions within the code itself, or driver layer that links to the code without altering the code being tested.

See also

  • Characterization test
    Characterization test
    In computer programming, a characterization test is a means to describe the actual behaviour of an existing piece of software, and therefore protect existing behaviour of legacy code against unintended changes via automated testing...

  • Quality control
    Quality control
    Quality control, or QC for short, is a process by which entities review the quality of all factors involved in production. This approach places an emphasis on three aspects:...

  • Software development process
    Software development process
    A software development process, also known as a software development life cycle , is a structure imposed on the development of a software product. Similar terms include software life cycle and software process. It is often considered a subset of systems development life cycle...

  • Software testing
    Software testing
    Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software...

  • Smoke testing

External links

The source of this article is wikipedia, the free encyclopedia.  The text of this article is licensed under the GFDL.
 
x
OK