TPT (Software)
Encyclopedia
TPT is a systematic test
methodology
for the automated
software test
and verification
of embedded control systems
. TPT is specialized on testing and validation embedded systems whose inputs and outputs can be represented as signals and is a dedicated method for continuous behavior of systems. Most control system
s belong to this system class. The outstanding characteristic of control systems is the fact that they interact closely interlinked with a real world environment. Controller need to observe their environment and react correspondingly to its behaviour. The system works in an interactional cycle with its environment and is subject to temporal constraints. Testing these systems is to stimulate and to check the timing
behavior. Traditional functional testing methods use scripts – TPT uses model-based testing
.
TPT combines a systematic and graphic modelling technique for test cases with a fully automatic test execution in different environments and automatic test evaluation. TPT covers the following four test activities:
texts as an element of the graphics support the simple and demonstrative readability even for non-programmers. Substantial techniques such as parallel and hierarchical state machines, conditional branching, signal description as well as measured
signals allow an intuitive and graphic modelling even of complex test cases.
generation, TPT even keeps track of very complex systems whose thorough testing requires a large amount of test cases thus making it possible to find failures in the system under test with an ideal amount of test cases.
The underlying idea of TPT’s systematic is the separation of similarities and differences among the test cases: most test cases are very similar in their structural process and can “only” be differentiated in a few, but crucial details. TPT makes use of this fact by jointly modelling and using joint structures. On the one hand, redundancies are thus avoided. On the other hand, it is made very clear what the test cases actually differ in – i.e. which specific aspect they respectively test. The comparability of test cases and thus the overview is improved in this approach and the attention of the tester is focused on the essential – the differentiating features of the test cases.
The hierarchical structure of the test cases makes it possible to break complex test problems down into sub-problems thus also improving the clarity and – as a result – the quality of the test.
These modelling techniques support the tester in finding the actually relevant cases, avoiding redundancies and keeping track of even large numbers of test cases.
in almost any environment due to the so called virtual machine (VM) concept. Examples are MATLAB
/Simulink
, TargetLink
, ASCET, C-Code, CAN
, LIN
, AUTOSAR
, LABCAR, INCA, Software-in-the-Loop (SiL) and HiL
. Thus TPT is an integrated tool to be used in all testing phases of the development like unit testing, integration testing
, system testing
and regression testing
.
(VM). The VM is the same for all platforms and all tests. Only a platform adapter
realizes the signal mapping for the individual application.
The TPT-VM is implemented in ANSI C
and requires a memory of just a few kilobytes and can completely do without a dynamic memory allocation, allowing it to be applied in minimalist and environments with few resources too.
TPT’s Virtual Machine is able to process tests in real time with defined response behaviour. The response times of TPT test cases are normally given within micro seconds – depending on the complexity and test hardware.
The offline evaluation is, technically speaking, based on the Python
script language, which has been extended by specific syntactic language elements and a specialized evaluation library in order to give optimal support to the test evaluation. The use of a script language ensures a high degree of flexibility in the test evaluation: access to reference data, communication with other tools and development of one’s own domain-specific libraries for test evaluation is supported. Besides of the script based test result evaluation user interfaces provide simple access to the test assessments and help non-programmers to avoid scripting.
Measurement data from other sources like TargetLink
and Simulink
signal logging or MCD-3 measurement data can be assessed automatically. This data can be independent from the test execution.
presents the result of the test evaluation to the tester in a HTML, report, in which not only the pure information “success”, “failed” or “unknown” can be depicted as the test result for each test case, but also details such as characteristic parameters or signals that have been observed in the test execution or computed in the test evaluation. Since the test assessment returns proper information about the timing and the checked behavior this information can be made available in the report.
The content of the test documentation as well as the structure of the document can be freely configured with the help of a template.
, DO-178B
, EN 50128 and ISO 26262
require traceability of requirements and tests
. TPT offers an interface to requirement
s tools like Telelogic
DOORS in order to support these activities.
tool and is applied mainly in the automotive controller development
and has originally been developed within Daimler AG for their own development. The first release of the test tool has been used in 2000. Daimler coordinated the development of the tool for years. Now PikeTec GmbH continues the development of the tool and is used by many different other car manufacturers and suppliers as Robert Bosch GmbH
, Continental and Hella
.
Software testing
Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software...
methodology
Methodology
Methodology is generally a guideline for solving a problem, with specificcomponents such as phases, tasks, methods, techniques and tools . It can be defined also as follows:...
for the automated
Test automation
Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions...
software test
Software testing
Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software...
and verification
Verification and Validation
In software project management, software testing, and software engineering, verification and validation is the process of checking that a software system meets specifications and that it fulfills its intended purpose...
of embedded control systems
Embedded system
An embedded system is a computer system designed for specific control functions within a larger system. often with real-time computing constraints. It is embedded as part of a complete device often including hardware and mechanical parts. By contrast, a general-purpose computer, such as a personal...
. TPT is specialized on testing and validation embedded systems whose inputs and outputs can be represented as signals and is a dedicated method for continuous behavior of systems. Most control system
Control system
A control system is a device, or set of devices to manage, command, direct or regulate the behavior of other devices or system.There are two common classes of control systems, with many variations and combinations: logic or sequential controls, and feedback or linear controls...
s belong to this system class. The outstanding characteristic of control systems is the fact that they interact closely interlinked with a real world environment. Controller need to observe their environment and react correspondingly to its behaviour. The system works in an interactional cycle with its environment and is subject to temporal constraints. Testing these systems is to stimulate and to check the timing
Timing
Timing is the time when something happens or the spacing of events in time. Some typical uses are:* The act of measuring the elapsed time of something or someone, often at athletic events such as swimming or running, where participants are timed with a device such as a stopwatch...
behavior. Traditional functional testing methods use scripts – TPT uses model-based testing
Model-based testing
Model-based testing is the application of Model based design for designing and optionally executing the necessary artifacts to perform software testing. Models can be used to represent the desired behavior of the System Under Test , or to represent the desired testing strategies and testing...
.
TPT combines a systematic and graphic modelling technique for test cases with a fully automatic test execution in different environments and automatic test evaluation. TPT covers the following four test activities:
- test case modelingModel-based testingModel-based testing is the application of Model based design for designing and optionally executing the necessary artifacts to perform software testing. Models can be used to represent the desired behavior of the System Under Test , or to represent the desired testing strategies and testing...
, - test execution in different environments (automated)
- test result analysis (test assessment (automated))
- test documentation (automated)
Graphic test cases
The exact process of individual test cases is modelled graphically with the aid of special state machines and time partitioning in TPT. Natural languageNatural language
In the philosophy of language, a natural language is any language which arises in an unpremeditated fashion as the result of the innate facility for language possessed by the human intellect. A natural language is typically used for communication, and may be spoken, signed, or written...
texts as an element of the graphics support the simple and demonstrative readability even for non-programmers. Substantial techniques such as parallel and hierarchical state machines, conditional branching, signal description as well as measured
Measurement
Measurement is the process or the result of determining the ratio of a physical quantity, such as a length, time, temperature etc., to a unit of measurement, such as the metre, second or degree Celsius...
signals allow an intuitive and graphic modelling even of complex test cases.
Systematic test cases
TPT was developed specifically for testing of continuous and reactive behaviour of embedded systems. TPT can be seen as the extension of the classification tree method in terms of timing behavior. Because of its systematic approach in test caseTest case
A test case in software engineering is a set of conditions or variables under which a tester will determine whether an application or software system is working correctly or not. The mechanism for determining whether a software program or system has passed or failed such a test is known as a test...
generation, TPT even keeps track of very complex systems whose thorough testing requires a large amount of test cases thus making it possible to find failures in the system under test with an ideal amount of test cases.
The underlying idea of TPT’s systematic is the separation of similarities and differences among the test cases: most test cases are very similar in their structural process and can “only” be differentiated in a few, but crucial details. TPT makes use of this fact by jointly modelling and using joint structures. On the one hand, redundancies are thus avoided. On the other hand, it is made very clear what the test cases actually differ in – i.e. which specific aspect they respectively test. The comparability of test cases and thus the overview is improved in this approach and the attention of the tester is focused on the essential – the differentiating features of the test cases.
The hierarchical structure of the test cases makes it possible to break complex test problems down into sub-problems thus also improving the clarity and – as a result – the quality of the test.
These modelling techniques support the tester in finding the actually relevant cases, avoiding redundancies and keeping track of even large numbers of test cases.
Reactive tests
With TPT, each test case can specifically react to the system’s behaviour during the testing process in real time – for instance in order to react on the system exactly when a certain system-state occurs or a sensor signal exceeds a certain threshold. If, for example, a sensor failure for an engine controller is to be simulated when the engine idling speed is exceeded, it has to be possible to react to the event “engine idling speed exceeded” in the description of the test case.Test execution
TPT test cases are made independent of its execution. The test cases can be executed in real timeReal-time computing
In computer science, real-time computing , or reactive computing, is the study of hardware and software systems that are subject to a "real-time constraint"— e.g. operational deadlines from event to system response. Real-time programs must guarantee response within strict time constraints...
in almost any environment due to the so called virtual machine (VM) concept. Examples are MATLAB
MATLAB
MATLAB is a numerical computing environment and fourth-generation programming language. Developed by MathWorks, MATLAB allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages,...
/Simulink
Simulink
Simulink, developed by MathWorks, is a commercial tool for modeling, simulating and analyzing multidomain dynamic systems. Its primary interface is a graphical block diagramming tool and a customizable set of block libraries. It offers tight integration with the rest of the MATLAB environment and...
, TargetLink
TargetLink
TargetLink is a software for automatic production code generation based on a subset of Simulink/Stateflow models by dSPACE GmbH. TargetLink requires an existing MATLAB/Simulink model....
, ASCET, C-Code, CAN
Controller Area Network
Controller–area network is a vehicle bus standard designed to allow microcontrollers and devices to communicate with each other within a vehicle without a host computer....
, LIN
Local Interconnect Network
The LIN-Bus is a vehicle bus standard or computer networking bus-system used within current automotive network architectures....
, AUTOSAR
AUTOSAR
AUTOSAR is an open and standardized automotive software architecture, jointly developed by automobile manufacturers, suppliers and tool developers...
, LABCAR, INCA, Software-in-the-Loop (SiL) and HiL
Hardware-in-the-loop simulation
Hardware-in-the-loop simulation is a technique that is used in the development and test of complex real-time embedded systems. HIL simulation provides an effective platform by adding the complexity of the plant under control to the test platform...
. Thus TPT is an integrated tool to be used in all testing phases of the development like unit testing, integration testing
Integration testing
Integration testing is the phase in software testing in which individual software modules are combined and tested as a group. It occurs after unit testing and before validation testing...
, system testing
System testing
System testing of software or hardware is testing conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements. System testing falls within the scope of black box testing, and as such, should require no knowledge of the inner design of the code or...
and regression testing
Regression testing
Regression testing is any type of software testing that seeks to uncover new errors, or regressions, in existing functionality after changes have been made to a system, such as functional enhancements, patches or configuration changes....
.
TPT virtual machine
The modeled test cases in TPT are compiled and during test execution interpreted by the so called virtual machineVirtual machine
A virtual machine is a "completely isolated guest operating system installation within a normal host operating system". Modern virtual machines are implemented with either software emulation or hardware virtualization or both together.-VM Definitions:A virtual machine is a software...
(VM). The VM is the same for all platforms and all tests. Only a platform adapter
Adapter
An adapter or adaptor is a person that adapts or a device that converts attributes of one device or system to those of an otherwise incompatible device or system.Some adapters may only affect physical attributes:...
realizes the signal mapping for the individual application.
The TPT-VM is implemented in ANSI C
ANSI C
ANSI C refers to the family of successive standards published by the American National Standards Institute for the C programming language. Software developers writing in C are encouraged to conform to the standards, as doing so aids portability between compilers.-History and outlook:The first...
and requires a memory of just a few kilobytes and can completely do without a dynamic memory allocation, allowing it to be applied in minimalist and environments with few resources too.
TPT’s Virtual Machine is able to process tests in real time with defined response behaviour. The response times of TPT test cases are normally given within micro seconds – depending on the complexity and test hardware.
Programmed test assessment
The expected system behaviour for individual test cases should also be automatically tested in order to assure efficient test processes. TPT offers the possibility to compute the properties for the expected behaviour online (during test execution) and offline (after test execution). While online evaluation uses the same modelling techniques as test modelling, offline evaluation offers decidedly more far-reaching possibilities for more complex evaluations, including operations such as comparisons with external reference data, limit-value monitoring, signal filters, analyses of state sequences and time conditions.The offline evaluation is, technically speaking, based on the Python
Python (programming language)
Python is a general-purpose, high-level programming language whose design philosophy emphasizes code readability. Python claims to "[combine] remarkable power with very clear syntax", and its standard library is large and comprehensive...
script language, which has been extended by specific syntactic language elements and a specialized evaluation library in order to give optimal support to the test evaluation. The use of a script language ensures a high degree of flexibility in the test evaluation: access to reference data, communication with other tools and development of one’s own domain-specific libraries for test evaluation is supported. Besides of the script based test result evaluation user interfaces provide simple access to the test assessments and help non-programmers to avoid scripting.
Measurement data from other sources like TargetLink
TargetLink
TargetLink is a software for automatic production code generation based on a subset of Simulink/Stateflow models by dSPACE GmbH. TargetLink requires an existing MATLAB/Simulink model....
and Simulink
Simulink
Simulink, developed by MathWorks, is a commercial tool for modeling, simulating and analyzing multidomain dynamic systems. Its primary interface is a graphical block diagramming tool and a customizable set of block libraries. It offers tight integration with the rest of the MATLAB environment and...
signal logging or MCD-3 measurement data can be assessed automatically. This data can be independent from the test execution.
Test documentation
TPT test documentation according to IEEE 829IEEE 829
IEEE 829-1998, also known as the 829 Standard for Software Test Documentation, is an IEEE standard that specifies the form of a set of documents for use in eight defined stages of software testing, each stage potentially producing its own separate type of document...
presents the result of the test evaluation to the tester in a HTML, report, in which not only the pure information “success”, “failed” or “unknown” can be depicted as the test result for each test case, but also details such as characteristic parameters or signals that have been observed in the test execution or computed in the test evaluation. Since the test assessment returns proper information about the timing and the checked behavior this information can be made available in the report.
The content of the test documentation as well as the structure of the document can be freely configured with the help of a template.
Requirements tracing
Industry norms such as IEC 61508IEC 61508
IEC 61508 is an international standard of rules applied in industry. It is titled "Functional Safety of Electrical/Electronic/Programmable Electronic Safety-related Systems"....
, DO-178B
DO-178B
DO-178B, Software Considerations in Airborne Systems and Equipment Certification is a document dealing with the safety of software used in airborne systems....
, EN 50128 and ISO 26262
ISO 26262
ISO 26262 is a Functional Safety standard , titled "Road vehicles -- Functional safety".This standard is an adaptation of the Functional Safety standard IEC 61508 for Automotive Electric/Electronic Systems.ISO 26262:...
require traceability of requirements and tests
Requirements traceability
Requirements traceability is a sub-discipline of requirements management within software development and systems engineering. Requirements traceability is concerned with documenting the life of a requirement and to provide bi-directional traceability between various associated requirements...
. TPT offers an interface to requirement
Requirement
In engineering, a requirement is a singular documented physical and functional need that a particular product or service must be or perform. It is most commonly used in a formal sense in systems engineering, software engineering, or enterprise engineering...
s tools like Telelogic
Telelogic
Telelogic AB was a software business headquartered in Malmö, Sweden. Telelogic was founded in 1983 as a research and development arm of Televerket, the Swedish department of telecom . It was later acquired by IBM Rational, and exists under the IBM software group.Telelogic had operations in 22...
DOORS in order to support these activities.
Application
TPT is a model-based testingModel-based testing
Model-based testing is the application of Model based design for designing and optionally executing the necessary artifacts to perform software testing. Models can be used to represent the desired behavior of the System Under Test , or to represent the desired testing strategies and testing...
tool and is applied mainly in the automotive controller development
Automotive software
Automotive software refers to any software, , related to automotive industry specific products or services.- See also :* Microsoft Auto* Infotainment* Telematics* *...
and has originally been developed within Daimler AG for their own development. The first release of the test tool has been used in 2000. Daimler coordinated the development of the tool for years. Now PikeTec GmbH continues the development of the tool and is used by many different other car manufacturers and suppliers as Robert Bosch GmbH
Robert Bosch GmbH
Robert Bosch GmbH is a multinational engineering and electronics company headquartered in Gerlingen, near Stuttgart, Germany. It is the world's largest supplier of automotive components...
, Continental and Hella
Hella (company)
Hella KGaA Hueck & Co. is an internationally operating German automotive part supplier with headquarters in Lippstadt, North Rhine-Westphalia. Core businesses are vehicle lighting and electronics systems and components. Hella is also involved in the areas of vehicle diagnostics and thermal...
.