Sanity test
Encyclopedia
A sanity test or sanity check is a basic test to quickly evaluate whether a claim or the result of a calculation can possibly be true. It is a simple check to see if the produced material is rational (that the material's creator was thinking rationally, applying sanity
). The point of a sanity test is to rule out certain classes of obviously false results, not to catch every possible error. A rule-of-thumb may be checked to perform the test. The advantage of a sanity test, over performing a complete or rigorous test, is speed.
In arithmetic, for example, when multiplying by 9, using the divisibility rule
for 9 to verify that the sum of digits
of the result is divisible by 9 is a sanity test - it will not catch every multiplication error, however it's a quick and simple method to discover many possible errors.
In computer science
, a sanity test is a very brief run-through of the functionality of a computer program
, system, calculation, or other analysis, to assure that part of the system or methodology works roughly as expected. This is often prior to a more exhaustive round of testing.
and other simple rule-of-thumb
devices applied to cross-check mathematical
calculations. For example:
which offers "quick, broad, and shallow testing") determines whether it is reasonable to proceed with further testing.
Software sanity tests are commonly conflated with smoke test
s. A smoke test determines whether it is possible to continue testing, as opposed to whether it is reasonable. A software smoke test determines whether the program launches and whether its interfaces are accessible and responsive (for example, the responsiveness of a web page or an input button). If the smoke test fails, it is impossible to conduct a sanity test. In contrast, the ideal sanity test exercises the smallest subset of application functions needed to determine whether the application logic is generally functional and correct (for example, an interest rate calculation for a financial application). If the sanity test fails, it is not reasonable to attempt more rigorous testing. Both sanity tests and smoke tests are ways to avoid wasting time and effort by quickly determining whether an application is too flawed to merit any rigorous testing. Many companies run sanity tests and unit test
s on an automated build
as part of their development process.
Sanity testing may be a tool used while manually debugging
software. An overall piece of software likely involves multiple subsystems between the input and the output. When the overall system is not working as expected, a sanity test can be used to make the decision on what to test next. If one subsystem is not giving the expected result, the other subsystems can be eliminated from further investigation until the problem with this one is solved.
The Hello world program
is often used as a sanity test for a development environment. If Hello World fails to compile or execute, the supporting environment likely has a configuration problem. If it works, the problem being diagnosed likely lies in the real application being diagnosed.
Another, possibly more common usage of 'sanity test' is to denote checks which are performed within program code, usually on arguments to functions or returns therefrom, to see if the answers can be assumed to be correct. The more complicated the routine, the more important that its response be checked. The trivial case is checking to see that a file opened, written to, or closed, did not fail on these activities – which is a sanity check often ignored by programmers. But more complex items can also be sanity-checked for various reasons.
Examples of this include bank account management systems which check that withdrawals are sane in not requesting more than the account contains, and that deposits or purchases are sane in fitting in with patterns established by historical data – large deposits may be more closely scrutinized for accuracy, large purchase transactions may be double-checked with a card holder for validity against fraud, ATM withdrawals in foreign locations never before visited by the card holder might be cleared up with him, etc.; these are "runtime" sanity checks, as opposed to the "development" sanity checks mentioned above.
Sanity
Sanity refers to the soundness, rationality and healthiness of the human mind, as opposed to insanity. A person is sane if they are rational...
). The point of a sanity test is to rule out certain classes of obviously false results, not to catch every possible error. A rule-of-thumb may be checked to perform the test. The advantage of a sanity test, over performing a complete or rigorous test, is speed.
In arithmetic, for example, when multiplying by 9, using the divisibility rule
Divisibility rule
A divisibility rule is a shorthand way of discovering whether a given number is divisible by a fixed divisor without performing the division, usually by examining its digits...
for 9 to verify that the sum of digits
Digit sum
In mathematics, the digit sum of a given integer is the sum of all its digits,...
of the result is divisible by 9 is a sanity test - it will not catch every multiplication error, however it's a quick and simple method to discover many possible errors.
In computer science
Computer science
Computer science or computing science is the study of the theoretical foundations of information and computation and of practical techniques for their implementation and application in computer systems...
, a sanity test is a very brief run-through of the functionality of a computer program
Computer program
A computer program is a sequence of instructions written to perform a specified task with a computer. A computer requires programs to function, typically executing the program's instructions in a central processor. The program has an executable form that the computer can use directly to execute...
, system, calculation, or other analysis, to assure that part of the system or methodology works roughly as expected. This is often prior to a more exhaustive round of testing.
Mathematical
A sanity test can refer to various orders of magnitudeOrder of magnitude
An order of magnitude is the class of scale or magnitude of any amount, where each class contains values of a fixed ratio to the class preceding it. In its most common usage, the amount being scaled is 10 and the scale is the exponent being applied to this amount...
and other simple rule-of-thumb
Rule of thumb
A rule of thumb is a principle with broad application that is not intended to be strictly accurate or reliable for every situation. It is an easily learned and easily applied procedure for approximately calculating or recalling some value, or for making some determination...
devices applied to cross-check mathematical
Mathematics
Mathematics is the study of quantity, space, structure, and change. Mathematicians seek out patterns and formulate new conjectures. Mathematicians resolve the truth or falsity of conjectures by mathematical proofs, which are arguments sufficient to convince other mathematicians of their validity...
calculations. For example:
- If one were to attempt to square 738 and calculated 53,874, a quick sanity check could show that this result cannot be true. Consider that 700 < 738, yet 700² = 7²×100² = 490000 > 53874. Since squaring positive numbers preserves their inequality, the result cannot be true, and so the calculated result is incorrect. The correct answer, 738² = 544,644, is more than 10 times higher than 53,874, and so the result had been off by an order of magnitude.
- In multiplication, 918 × 155 is not 142135 since 918 is divisible by three but 142135 is not (digits add up to 16, not a multiple of three). Also, the product must end in the same digit as the product of end-digits 8×5=40, but 142135 does not end in "0" like "40", while the correct answer does: 918×155=142290. An even quicker check is that the product of even and odd numbers is even, whereas 142135 is odd.
- When talking about quantities in physicsPhysicsPhysics is a natural science that involves the study of matter and its motion through spacetime, along with related concepts such as energy and force. More broadly, it is the general analysis of nature, conducted in order to understand how the universe behaves.Physics is one of the oldest academic...
, the powerPower (physics)In physics, power is the rate at which energy is transferred, used, or transformed. For example, the rate at which a light bulb transforms electrical energy into heat and light is measured in watts—the more wattage, the more power, or equivalently the more electrical energy is used per unit...
output of a carAutomobileAn automobile, autocar, motor car or car is a wheeled motor vehicle used for transporting passengers, which also carries its own engine or motor...
cannot be 700 kJ since that is a unit of energyEnergyIn physics, energy is an indirectly observed quantity. It is often understood as the ability a physical system has to do work on other physical systems...
, not power (energy per unit time). See dimensional analysisDimensional analysisIn physics and all science, dimensional analysis is a tool to find or check relations among physical quantities by using their dimensions. The dimension of a physical quantity is the combination of the basic physical dimensions which describe it; for example, speed has the dimension length per...
.
Software development
In software development, the sanity test (a form of software testingSoftware testing
Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software...
which offers "quick, broad, and shallow testing") determines whether it is reasonable to proceed with further testing.
Software sanity tests are commonly conflated with smoke test
Smoke test
Smoke testing refers to physical tests made to closed systems of pipes to test for leaks. By metaphorical extension, the term is also used for the first test made after assembly or repairs to a system, to provide some assurance that the system under test will not catastrophically fail...
s. A smoke test determines whether it is possible to continue testing, as opposed to whether it is reasonable. A software smoke test determines whether the program launches and whether its interfaces are accessible and responsive (for example, the responsiveness of a web page or an input button). If the smoke test fails, it is impossible to conduct a sanity test. In contrast, the ideal sanity test exercises the smallest subset of application functions needed to determine whether the application logic is generally functional and correct (for example, an interest rate calculation for a financial application). If the sanity test fails, it is not reasonable to attempt more rigorous testing. Both sanity tests and smoke tests are ways to avoid wasting time and effort by quickly determining whether an application is too flawed to merit any rigorous testing. Many companies run sanity tests and unit test
Unit test
In computer programming, unit testing is a method by which individual units of source code are tested to determine if they are fit for use.A unit is the smallest testable part of an application. In procedural programming a unit could be an entire module but is more commonly an individual function...
s on an automated build
Build Automation
Build automation is the act of scripting or automating a wide variety of tasks that software developers do in their day-to-day activities including things like:* compiling computer source code into binary code* packaging binary code* running tests...
as part of their development process.
Sanity testing may be a tool used while manually debugging
Debugging
Debugging is a methodical process of finding and reducing the number of bugs, or defects, in a computer program or a piece of electronic hardware, thus making it behave as expected. Debugging tends to be harder when various subsystems are tightly coupled, as changes in one may cause bugs to emerge...
software. An overall piece of software likely involves multiple subsystems between the input and the output. When the overall system is not working as expected, a sanity test can be used to make the decision on what to test next. If one subsystem is not giving the expected result, the other subsystems can be eliminated from further investigation until the problem with this one is solved.
The Hello world program
Hello world program
A "Hello world" program is a computer program that outputs "Hello world" on a display device. Because it is typically one of the simplest programs possible in most programming languages, it is by tradition often used to illustrate to beginners the most basic syntax of a programming language, or to...
is often used as a sanity test for a development environment. If Hello World fails to compile or execute, the supporting environment likely has a configuration problem. If it works, the problem being diagnosed likely lies in the real application being diagnosed.
Another, possibly more common usage of 'sanity test' is to denote checks which are performed within program code, usually on arguments to functions or returns therefrom, to see if the answers can be assumed to be correct. The more complicated the routine, the more important that its response be checked. The trivial case is checking to see that a file opened, written to, or closed, did not fail on these activities – which is a sanity check often ignored by programmers. But more complex items can also be sanity-checked for various reasons.
Examples of this include bank account management systems which check that withdrawals are sane in not requesting more than the account contains, and that deposits or purchases are sane in fitting in with patterns established by historical data – large deposits may be more closely scrutinized for accuracy, large purchase transactions may be double-checked with a card holder for validity against fraud, ATM withdrawals in foreign locations never before visited by the card holder might be cleared up with him, etc.; these are "runtime" sanity checks, as opposed to the "development" sanity checks mentioned above.
See also
- Proof of conceptProof of conceptA proof of concept or a proof of principle is a realization of a certain method or idea to demonstrate its feasibility, or a demonstration in principle, whose purpose is to verify that some concept or theory that has the potential of being used...
- Back-of-the-envelope calculationBack-of-the-envelope calculationA back-of-the-envelope calculation is a rough calculation, typically jotted down on any available scrap of paper such as the actual back of an envelope...
- Software testingSoftware testingSoftware testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software...
- Mental calculationMental calculationMental calculation comprises arithmetical calculations using only the human brain, with no help from calculators, computers, or pen and paper. People use mental calculation when computing tools are not available, when it is faster than other means of calculation , or in a competition context...
- Order of magnitudeOrder of magnitudeAn order of magnitude is the class of scale or magnitude of any amount, where each class contains values of a fixed ratio to the class preceding it. In its most common usage, the amount being scaled is 10 and the scale is the exponent being applied to this amount...
- Fermi problemFermi problemIn science, particularly in physics or engineering education, a Fermi problem, Fermi question, or Fermi estimate is an estimation problem designed to teach dimensional analysis, approximation, and the importance of clearly identifying one's assumptions...