In software testing

Software testing

Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software...

, stress testing

Stress testing

Stress testing is a form of testing that is used to determine the stability of a given system or entity. It involves testing beyond normal operational capacity, often to a breaking point, in order to observe the results...

refers to tests that determine the robustness of software by testing beyond the limits of normal operation. Stress testing is particularly important for "mission critical

Mission Critical

Mission critical refers to any factor of a system whose failure will result in the failure of business operations. That is, it is critical to the organization's 'mission'....

" software, but is used for all types of software. Stress tests commonly put a greater emphasis on robustness, availability

Availability

In telecommunications and reliability theory, the term availability has the following meanings:* The degree to which a system, subsystem, or equipment is in a specified operable and committable state at the start of a mission, when the mission is called for at an unknown, i.e., a random, time...

, and error handling under a heavy load, than on what would be considered correct behavior under normal circumstances.

Field experience

Failures may be related to:

• use of non production like environments, e.g. databases of smaller size
• complete lack of load
  Load testing
  Load testing is the process of putting demand on a system or device and measuring its response. Load testing is performed to determine a system’s behavior under both normal and anticipated peak load conditions. It helps to identify the maximum operating capacity of an application as well as any...
  
   or stress testing

Rationale

Reasons for stress testing include:

• The software being tested is "mission critical", that is, failure of the software (such as a crash) would have disastrous consequences.
• The amount of time and resources dedicated to testing is usually not sufficient, with traditional testing methods, to test all of the situations in which the software will be used when it is released.
• Even with sufficient time and resources for writing tests, it may not be possible to determine before hand all of the different ways in which the software will be used. This is particularly true for operating system
  Operating system
  An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
  
  s and middleware
  Middleware
  Middleware is computer software that connects software components or people and their applications. The software consists of a set of services that allows multiple processes running on one or more machines to interact...
  
  , which will eventually be used by software that doesn't even exist at the time of the testing.
• Customers may use the software on computers that have significantly fewer computational resources (such as memory or disk space
  Disk storage
  Disk storage or disc storage is a general category of storage mechanisms, in which data are digitally recorded by various electronic, magnetic, optical, or mechanical methods on a surface layer deposited of one or more planar, round and rotating disks...
  
  ) than the computers used for testing.
• Concurrency
  Concurrency (computer science)
  In computer science, concurrency is a property of systems in which several computations are executing simultaneously, and potentially interacting with each other...
  
   is particularly difficult to test with traditional testing methods. Stress testing may be necessary to find race condition
  Race condition
  A race condition or race hazard is a flaw in an electronic system or process whereby the output or result of the process is unexpectedly and critically dependent on the sequence or timing of other events...
  
  s and deadlock
  Deadlock
  A deadlock is a situation where in two or more competing actions are each waiting for the other to finish, and thus neither ever does. It is often seen in a paradox like the "chicken or the egg"...
  
  s.
• Software such as web server
  Web server
  Web server can refer to either the hardware or the software that helps to deliver content that can be accessed through the Internet....
  
  s that will be accessible over the Internet
  Internet
  The Internet is a global system of interconnected computer networks that use the standard Internet protocol suite to serve billions of users worldwide...
  
   may be subject to denial of service attacks.
• Under normal conditions, certain types of bugs
  Software bug
  A software bug is the common term used to describe an error, flaw, mistake, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways. Most bugs arise from mistakes and errors made by people in either a program's...
  
  , such as memory leak
  Memory leak
  A memory leak, in computer science , occurs when a computer program consumes memory but is unable to release it back to the operating system. In object-oriented programming, a memory leak happens when an object is stored in memory but cannot be accessed by the running code...
  
  s, can be fairly benign and difficult to detect over the short periods of time in which testing is performed. However, these bugs can still be potentially serious. In a sense, stress testing for a relatively short period of time can be seen as simulating normal operation for a longer period of time.

Relationship to branch coverage

Branch

Branch (computer science)

A branch is sequence of code in a computer program which is conditionally executed depending on whether the flow of control is altered or not . The term can be used when referring to programs in high level languages as well as program written in machine code or assembly language...

 coverage (a specific type of code coverage

Code coverage

Code coverage is a measure used in software testing. It describes the degree to which the source code of a program has been tested. It is a form of testing that inspects the code directly and is therefore a form of white box testing....

) is a metric of the number of branches executed under test, where "100% branch coverage" means that every branch in a program has been executed at least once under some test. Branch coverage is one of the most important metrics for software testing; software for which the branch coverage is low is not generally considered to be thoroughly tested. Note that code coverage metrics are a property of the tests for a piece of software, not of the software being tested.

Achieving high branch coverage often involves writing negative test variations, that is, variations where the software is supposed to fail in some way, in addition to the usual positive test variations, which test intended usage. An example of a negative variation would be calling a function with illegal parameters. There is a limit to the branch coverage that can be achieved even with negative variations, however, as some branches may only be used for handling of errors that are beyond the control of the test. For example, a test would normally have no control over memory allocation, so branches that handle an "out of memory" error are difficult to test.

Stress testing can achieve higher branch coverage by producing the conditions under which certain error handling branches are followed. The coverage can be further improved by using fault injection

Fault injection

In software testing, fault injection is a technique for improving the coverage of a test by introducing faults to test code paths, in particular error handling code paths, that might otherwise rarely be followed. It is often used with stress testing and is widely considered to be an important part...

.

Examples

• A web server
  Web server
  Web server can refer to either the hardware or the software that helps to deliver content that can be accessed through the Internet....
  
   may be stress tested using scripts
  Shell script
  A shell script is a script written for the shell, or command line interpreter, of an operating system. It is often considered a simple domain-specific programming language...
  
  , bots
  Internet bot
  Internet bots, also known as web robots, WWW robots or simply bots, are software applications that run automated tasks over the Internet. Typically, bots perform tasks that are both simple and structurally repetitive, at a much higher rate than would be possible for a human alone...
  
  , and various denial of service tools to observe the performance of a web site during peak loads.