A test execution engine is a type of software used to test software

Computer software

Computer software, or just software, is a collection of computer programs and related data that provide the instructions for telling a computer what to do and how to do it....

, hardware

Computer hardware

Personal computer hardware are component devices which are typically installed into or peripheral to a computer case to create a personal computer upon which system software is installed including a firmware interface such as a BIOS and an operating system which supports application software that...

 or complete systems.

Synonyms of test execution engine:

• Test executive
• Test manager

A test execution engine may appear in two forms:

• Module
  Module
  Module or modular may refer to the concept of modularity. It may also refer to:-Computing and engineering:* Modular design, the engineering discipline of designing complex devices using separately designed sub-components...
  
   of a test software suite (test bench
  Test bench
  A test bench is a virtual environment used to verify the correctness or soundness of a design or model .The term has its roots in the testing of electronic devices, where an engineer would sit at a lab bench with tools for measurement and manipulation, such as oscilloscopes, multimeters, soldering...
  
  ) or an integrated development environment
  Integrated development environment
  An integrated development environment is a software application that provides comprehensive facilities to computer programmers for software development...
  
  
• Stand-alone application software
  Application software
  Application software, also known as an application or an "app", is computer software designed to help the user to perform specific tasks. Examples include enterprise software, accounting software, office suites, graphics software and media players. Many application programs deal principally with...
  
  

Concept

The test execution engine does not carry any information about the tested product. Only the test specification and the test data carries information about the tested product.

The test specification is software. Test specification is sometimes referred to as test sequence, which consists of test steps.

The test specification should be stored in the test repository in a text format (such as source code

Source code

In computer science, source code is text written using the format and syntax of the programming language that it is being written in. Such a language is specially designed to facilitate the work of computer programmers, who specify the actions to be performed by a computer mostly by writing source...

).
Test data is sometimes generated by some test data generator tool.
Test data can be stored in binary or text files.
Test data should also be stored in the test repository together with the test specification.

Test specification is selected, loaded and executed by the test execution engine similarly, as application software

Application software

Application software, also known as an application or an "app", is computer software designed to help the user to perform specific tasks. Examples include enterprise software, accounting software, office suites, graphics software and media players. Many application programs deal principally with...

 is selected, loaded and executed by operation systems. The test execution engine should not operate on the tested object directly, but though plug-in modules similarly as an application software

Application software

Application software, also known as an application or an "app", is computer software designed to help the user to perform specific tasks. Examples include enterprise software, accounting software, office suites, graphics software and media players. Many application programs deal principally with...

 accesses devices through driver

Device driver

In computing, a device driver or software driver is a computer program allowing higher-level computer programs to interact with a hardware device....

s which are installed on the operation system.

The difference between the concept of test execution engine and operation system is that the test execution engine monitors, presents and stores the status, results, time stamp, length and other information for every Test Step of a Test Sequence, but typically an operation system does not perform such profiling

Profiling

Profiling, the extrapolation of information about something, based on known qualities, may refer specifically to:* Profiling practices * Forensic profiling, used in several types of forensic sciences* Offender profiling...

 of a software execution.

Reasons for using a test execution engine:

• Test results are stored and can be viewed in a uniform way, independent of the type of the test
• Easier to keep track of the changes
• Easier to reuse components developed for testing

Functions

Main functions of a test execution engine:

• Select a test type to execute. Selection can be automatic or manual.
• Load the specification of the selected test type by opening a file from the local file system
  File system
  A file system is a means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device which contain it. A file system organizes data in an efficient manner and is tuned to the...
  
   or downloading it from a Server
  Server (computing)
  In the context of client-server architecture, a server is a computer program running to serve the requests of other programs, the "clients". Thus, the "server" performs some computational task on behalf of "clients"...
  
  , depending on where the test repository is stored.
• Execute the test through the use of testing tools (SW test) or instruments (HW test), while showing the progress and accepting control from the operator (for example to Abort
  Abort (computing)
  In a computer or data transmission system, to abort means to terminate, usually in a controlled manner, a processing activity because it is impossible or undesirable for the activity to proceed. Such an action may be accompanied by diagnostic information on the aborted process.In addition to being...
  
  )
• Present the outcome (such as Passed, Failed or Aborted) of test Steps and the complete Sequence to the operator
• Store the Test Results in report files

An advanced test execution engine may have additional functions, such as:

• Store the test results in a Database
  Database
  A database is an organized collection of data for one or more purposes, usually in digital form. The data are typically organized to model relevant aspects of reality , in a way that supports processes requiring this information...
  
  
• Load test result back from the Database
  Database
  A database is an organized collection of data for one or more purposes, usually in digital form. The data are typically organized to model relevant aspects of reality , in a way that supports processes requiring this information...
  
  
• Present the test results as raw data.
• Present the test results in a processed format. (Statistics
  Statistics
  Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
  
  )
• Authenticate the operators.

Advanced functions of the test execution engine maybe less important for software testing, but these advanced features could be essential when executing hardware/system tests.

Operations types

A test execution engine by executing a test specification, it may perform different types of operations on the product, such as:

• Verification
  Verification and Validation
  In software project management, software testing, and software engineering, verification and validation is the process of checking that a software system meets specifications and that it fulfills its intended purpose...
  
  
• Calibration
  Calibration
  Calibration is a comparison between measurements – one of known magnitude or correctness made or set with one device and another measurement made in as similar a way as possible with a second device....
  
  
• Programming
  • Downloading firmware
    Firmware
    In electronic systems and computing, firmware is a term often used to denote the fixed, usually rather small, programs and/or data structures that internally control various electronic devices...
    
     to the product's nonvolatile memory (Flash
    Flash memory
    Flash memory is a non-volatile computer storage chip that can be electrically erased and reprogrammed. It was developed from EEPROM and must be erased in fairly large blocks before these can be rewritten with new data...
    
    )
  • Personalization
    Personalization
    Personalization involves using technology to accommodate the differences between individuals. Once confined mainly to the Web, it is increasingly becoming a factor in education, health care Personalization involves using technology to accommodate the differences between individuals. Once confined...
    
    : programming with unique parameters, like a serial number
    Serial number
    A serial number is a unique number assigned for identification which varies from its successor or predecessor by a fixed discrete integer value...
    
     or a MAC address
    MAC address
    A Media Access Control address is a unique identifier assigned to network interfaces for communications on the physical network segment. MAC addresses are used for numerous network technologies and most IEEE 802 network technologies, including Ethernet...
    
    

If the subject is a software, verification is the only possible operation.

Proprietary

Software test:

• IBM's IBM Rational Quality Manager

Hardware or system test:

• National Instruments
  National Instruments
  National Instruments Corporation, or NI , is an American company with over 5,000 employees and direct operations in 41 countries. Headquartered in Austin, Texas, it is a producer of automated test equipment and virtual instrumentation software...
  
  ' TestStand - Test Management Software
• Hiatronics' Hiatronic Development Suite - Test Stand Content Management System
• Geotest's ATEasy - Rapid Application Development Framework

Open Source

Hardware or system test:

• JTStand - Scripting Environment for Data Collection
• bumbleTest - Framework for executing and recording the results of tests and benchmarks