[ Home | What's New | Contents | Overview | Contributors | Distribution | Examples | Documentation | Manual | Publications | Mailing List Archive | Problems ] This page was last updated by Axel Belinfante on 2006-06-30
TorX Test Tool Information
Prev   Next

torx(1) - execute test on-the-fly

Table of Contents

Name

torx - execute test on-the-fly

Synopsis

torx [ options ... ] configuration-file ...
torx --help

Description

torx reads the given configuration file(s), starts the primer and adaptor specified in the configuration file(s) (see torx-config(4)), after which it either starts on-the-fly test generation and execution (when the --depth option was given) or prints its prompt (tester> ) on standard output and waits for user commands from standard input. When torx detects an error, it prints a Fail verdict together with the the list of expected output events to standard output and exits. The output of the commands given to torx is written to standard output; diagnostics of the primer and adaptor are printed to standard error.

To be added: the command line arguments with which primer and adaptor are invoked.

A basic concept in torx is the basic test action or event. As representation of an event torx basically uses LOTOS-like notation (consisting of a gate name followed by zero or more value-expressions, where each value-expression is preceded by an exclamation mark (!)), with the extension that it may contain variables in the value-expressions. These variables have the following form: var_type$nr where the $nr part is optional. Here type is the type of the variable, and nr is a sequence number to make variable occurences unique in the event. The special event suspension is represented by (Suspension).

Each test step that torx executes (as result of a command, or during execution in automatic mode) is reported on standard output. Each test step appears on a separate line, containing from left to right: the test step number, the iokind (input or output), an opening parenthesis (() the channel name, a colon (:), the pco name, a closing parenthesis ()), a colon (:), followed by the event. For suspension events the pco name and preceding colon are omitted.

Options

Apart from the --help option, all torx options that can be set from the command line can also be set as configuration file option (see torx-config(4)). The command line options overrule settings specified in a configuration file.

The following command line options are supported:

--help
print the version number and an overview of the command line options, and exit.
--batch
start the tester in batch mode. In this mode no adapter is started, but a batch test in Aldebaran (.aut) format is generated on standard output, of depth as given with the --depth flag. By default, this will have the format of a tree (i.e. there will be no cycles) This is rather new and not very well tested!
--batch-automaton
generate the batch test in the form of an automaton (may contain cycles). Note: this flag does not imply the --batch flag which must be separately given.
--depth nr
(config: MAXSTEPS) start the tester in automatic mode, try to execute nr of test events, and exit when done. The automatic execution can be interrupted by giving a stop command, which causes a prompt to be printed.
--no-depth
(config: MAXSTEPS) start torx in manual mode, with unlimited depth (this is the default)
--seed nr
(config: SEED) use nr as seed for random number generator(s)
--no-seed
(config: SEED) use random seed (based on current time) (this is the default)
--log file
(config: LOGFILE) write log to file. If file already exists torx will extend the file name to be unique, by extending it with the string .~n~ where n is the smallest number (from 0) that makes the file name unique.
--no-log
(config: LOGFILE) do not write a log-file (this is the default)
--logmon command
(config: LOGMON) start torx log monitoring command as background process that can continue to run even after torx itself has exited, and write log to its standard input. Multiple --logmon command line options may be given, to start multiple commands (e.g. to use multiple viewers). The connection to command is made via the utility program torx-logclient(1).
--no-logmon
(config: LOGMON) do not start a log monitoring command (this is the default)
--trace file
(config: TRACEFILE) read trace from file.
--no-trace
(config: TRACEFILE) do not use a trace-file (this is the default)

Commands

The commands of torx are grouped in three sections: 1) general ones, 2) commands that give information about the current test execution status, without changing the current execution `state', and 3) commands that execute a (sequence of) testing step(s).

general

help
print an overview of the recognized commands.
quit
(clean up and) exit the tester

informational

The following commands only print information, without doing a test step. These commands do not cause a state change in primer or adaptor.
path
print the path (the events done from start till now)
menu
print the menu (distinquishes inputs from outputs). Each menu-element is printed on a separate line, containing from left to right: the iokind (input or output), followed by a colon (:), the channel name between parentheses, followed by the event.
trace
print the current event from the trace that we follow
state
print the state(vector) (internal format of the primer)
menusize
print the size of the menu (this command will probably be deprecated)
statesize
print the size of the state(vector) (internal format) (this command will probably be deprecated)

execution

The following commands (may) cause the execution of a testing step. Some commands can only be executed if the (parenthesized) condition at the start of their explanation below holds. If torx is unable to execute a command, it will print an error message to standard output, and issue a new prompt. Note: in the commands next, step, input, and output, the parentheses around the channel name argument are part of the command syntax and can not be omitted.
io
select randomly input or output, to be used in next step
next (chan) input-event
(input-event isin menu) do one step using input-event
next
do one step (using the result of last io command)
step (chan) input-event
synonym for next (chan) event
step
synonym for next
input (chan) event
(event isin menu) do one input step using event
input event
(event isin menu) do one input step using event
input (chan)
(menu of chan is non-empty) do one input step, from channel chan
input
(input menu is non-empty) do one input step, from randomly chosen channel
output (chan)
do one output step, from channel chan
output
do one output step, from randomly chosen channel
auto
switch to automatic mode: do steps, randomly choosing in- and outputs until end of test, or until interrupted by the stop command
auto nr
as auto, but do at most nr steps
usetrace
use the current trace event for next step
autotrace
switch to automatic mode: do steps, following the trace, until end of test, or end of trace, or until interrupted by the stop command
autotrace nr
as autotrace, but do at most nr steps
stop
interrupt the auto or autotrace command and print a prompt.

Example

Below we show a sh(1) shell script that demonstrates how torx can be used in `batch' mode to repeatedly execute tests upto a given number of test steps, for a given set of mutants, using a different random number seed in each execution run. We assume here that the mutant can be selected by setting the variable MUTANT in the environment. The command used to invoke torx is split-up over several lines for clarity.

#!/bin/sh
first=1
beyond=1000
depth=1000000
mutants="111 222 333 444 555 666 777 888 999 000"
export MUTANT
i=$first
while test $i -lt $beyond
do
    for m in $mutants
    do
        MUTANT=$m
        torx --depth $depth \
            --seed $i \
            --log testloop.$i.$m.log \
             config.if \
            > testloop.$i.$m.out 2>&1
        sleep 60
    done
    i=`expr $i + 1`
done
The assumption behind this script is that the implementation under test will be started by (a shell script started by) torx (actually: by the torx adaptor) which means that it (the implementation resp. the shell script) will see variable MUTANT in its environment, and act on it.

See Also

torx-intro(1), mkprimer(1), torx-logclient(1), torx-primer(5), torx-adaptor(5), torx-config(4), torx-log(4), xtorx(1), sh(1)

Diagnostics

The diagnostic messages of primer and adaptor are passed on to the standard error. If an error is encountered in an command given to torx, an appropriate error message is given (on standard output), and a new prompt is printed. The error messages should be self-explanatory.

Return-value

In case of normal termination (whether or not an error is found) torx always returns with a 0 exit status. A non-zero exit status will only be given when an (unforeseen?) internal or external error makes normal termination impossible.

Bugs

The diagnostics of primer and adaptor appear interspersed with the output of torx; giving torx an empty command (just press return) prints a new prompt.

Using a syntactically wrong event as argument to a command will cause torx to exit.

The --depth flag should be treated in a slightly different way: after automatically doing the test steps required by this flag, torx should execute commands given on standard input, until end-of-file on standard input, or until a quit command is given. (however, this would require batch scripts to be updated, to invoke torx with standard input redirected from /dev/null)

Contact

By Email: <torx_support@cs.utwente.nl>

Version

This manual page documents version 3.9.0 of torx.


Table of Contents


Prev Table of Contents Next
Appendix D: TorX Manual Pages: torx-sendpr(1) - problem report utility Valid HTML 4.01! Appendix D: TorX Manual Pages: torx_open(1) - run open/caesar tool on torx explorer program