TestU01
Encyclopedia
TestU01 is a software library, implemented in the ANSI C
language, and offering a collection of utilities for the empirical statistical testing of uniform random number generators.
The library implements several types of random number generators in generic form, as well as many specific generators proposed in the literature or found in widely-used software. It provides general implementations of the classical statistical tests for random number generators, as well as several others proposed in the literature, and some original ones. These tests can be applied to the generators predefined in the library and to user-defined generators. Specific tests suites for either sequences of uniform random numbers in [0,1] or bit sequences are also available. Basic tools for plotting vectors of points produced by generators are provided as well.
It provides general implementations of the classical statistical tests for RNGs, as well as several others tests proposed in the literature, and some original ones. Predefined tests suites for sequences of uniform random numbers over the interval (0, 1) and for bit sequences are available. Tools are also offered to perform systematic studies of the interaction between a specific test and the structure of the point sets produced by a given family of RNGs. That is, for a given kind of test and a given class of RNGs, to determine how large should be the sample size of the test, as a function of the generator's period length, before the generator starts to fail the test systematically. Finally, the library provides various types of generators implemented in generic form, as well as many specific generators proposed in the literature or found in widely used software. The tests can be applied to instances of the generators predefined in the library, or to user-defined generators, or to streams of random numbers produced by any kind of device or stored in files.
's tests were supplanted by George Marsaglia
's (1996) DIEHARD tests, and DIEHARD has been the standard for several years.
To use Marsaglia's program, the user creates a file of three million random numbers, and the program analyzes these numbers. There are some notable difficulties with DIEHARD.
First, it is not user-friendly.
Second, it does not offer many tests—about 15.
Third, the parameters of the tests are fixed, and it is often
advantageous to vary them—see the online appendix to this article at the JAE Data Archive for a demonstration of this point.
Fourth, it is not extensible—new tests cannot be added.
Fifth, while these tests might have been stringent 10 years ago, they are not now.
When a specific test is applied to a sample of size n produced by an RNG, the p-value of the test usually will remain ‘reasonable’ as the sample size increases until the sample size hits n0, say. After that, the p-value diverges to 0 or 1 with exponential speed. Module 4 allows the researcher to investigate how large the sample size should be, as a function of the RNG’s period, before the RNG starts to fail the test systematically.
As far as testing is concerned, most users who are not specialists in random numbers will not want to choose from among the many statistical tests available, set parameters for each one and apply them serially; i.e., most users will not be concerned with (ii) above. Rather, most users will want to use the per-programmed batteries of tests (iii) to test the RNGs that are used in their statistical packages. To satisfy the needs of such users, TESTU01 offers three batteries of tests called ‘Small Crush’ (which consists of 10 tests), ‘Crush’ (60 tests) and ‘Big Crush’ (45 tests).4
The specific tests applied by each battery are given in the User Guide.5 On a 1.7 GHz Pentium 4
running Red Hat Linux 9.0, for a simple RNG, Small Crush takes about 2 minutes. Crush takes about 1.7 hours. Big Crush takes about 12 hours. (For a more complex RNG, all these times
increase by a factor of two or more.) By contrast, the DIEHARD tests take about 15 seconds to run.
ANSI C
ANSI C refers to the family of successive standards published by the American National Standards Institute for the C programming language. Software developers writing in C are encouraged to conform to the standards, as doing so aids portability between compilers.-History and outlook:The first...
language, and offering a collection of utilities for the empirical statistical testing of uniform random number generators.
The library implements several types of random number generators in generic form, as well as many specific generators proposed in the literature or found in widely-used software. It provides general implementations of the classical statistical tests for random number generators, as well as several others proposed in the literature, and some original ones. These tests can be applied to the generators predefined in the library and to user-defined generators. Specific tests suites for either sequences of uniform random numbers in [0,1] or bit sequences are also available. Basic tools for plotting vectors of points produced by generators are provided as well.
Introduction
TestU01, a software library implemented in the ANSI C language, and offering a collection of utilities for the empirical statistical testing of uniform random number generators (RNGs).It provides general implementations of the classical statistical tests for RNGs, as well as several others tests proposed in the literature, and some original ones. Predefined tests suites for sequences of uniform random numbers over the interval (0, 1) and for bit sequences are available. Tools are also offered to perform systematic studies of the interaction between a specific test and the structure of the point sets produced by a given family of RNGs. That is, for a given kind of test and a given class of RNGs, to determine how large should be the sample size of the test, as a function of the generator's period length, before the generator starts to fail the test systematically. Finally, the library provides various types of generators implemented in generic form, as well as many specific generators proposed in the literature or found in widely used software. The tests can be applied to instances of the generators predefined in the library, or to user-defined generators, or to streams of random numbers produced by any kind of device or stored in files.
History
An initial battery of statistical tests for uniform RNGs was offered by the 1969 first edition of Knuth (1997). In popular testing of RNGs, Donald KnuthDonald Knuth
Donald Ervin Knuth is a computer scientist and Professor Emeritus at Stanford University.He is the author of the seminal multi-volume work The Art of Computer Programming. Knuth has been called the "father" of the analysis of algorithms...
's tests were supplanted by George Marsaglia
George Marsaglia
George Marsaglia was an American mathematician and computer scientist. He established the lattice structure of congruential random number generators in the paper "Random numbers fall mainly in the planes". This phenomenon is sometimes called the Marsaglia effect...
's (1996) DIEHARD tests, and DIEHARD has been the standard for several years.
To use Marsaglia's program, the user creates a file of three million random numbers, and the program analyzes these numbers. There are some notable difficulties with DIEHARD.
First, it is not user-friendly.
Second, it does not offer many tests—about 15.
Third, the parameters of the tests are fixed, and it is often
advantageous to vary them—see the online appendix to this article at the JAE Data Archive for a demonstration of this point.
Fourth, it is not extensible—new tests cannot be added.
Fifth, while these tests might have been stringent 10 years ago, they are not now.
Review
TESTU01 offers four groups of modules for analyzing RNGs:- Implementing (pre-programmed)RNGs;
- implementing specific statistical tests;
- implementing batteries of statistical tests; and
- Applying tests to entire families of RNGs.
When a specific test is applied to a sample of size n produced by an RNG, the p-value of the test usually will remain ‘reasonable’ as the sample size increases until the sample size hits n0, say. After that, the p-value diverges to 0 or 1 with exponential speed. Module 4 allows the researcher to investigate how large the sample size should be, as a function of the RNG’s period, before the RNG starts to fail the test systematically.
As far as testing is concerned, most users who are not specialists in random numbers will not want to choose from among the many statistical tests available, set parameters for each one and apply them serially; i.e., most users will not be concerned with (ii) above. Rather, most users will want to use the per-programmed batteries of tests (iii) to test the RNGs that are used in their statistical packages. To satisfy the needs of such users, TESTU01 offers three batteries of tests called ‘Small Crush’ (which consists of 10 tests), ‘Crush’ (60 tests) and ‘Big Crush’ (45 tests).4
The specific tests applied by each battery are given in the User Guide.5 On a 1.7 GHz Pentium 4
running Red Hat Linux 9.0, for a simple RNG, Small Crush takes about 2 minutes. Crush takes about 1.7 hours. Big Crush takes about 12 hours. (For a more complex RNG, all these times
increase by a factor of two or more.) By contrast, the DIEHARD tests take about 15 seconds to run.