torx(1) - execute test on-the-fly
Table of Contents
torx - execute test on-the-fly
torx [ options ... ] configuration-file
...
torx --help
torx reads the given configuration file(s), starts
the primer and adaptor specified in the configuration file(s) (see torx-config(4)),
after which it either starts on-the-fly test generation and execution (when
the --depth option was given) or prints its prompt (tester> ) on standard
output and waits for user commands from standard input. When torx detects
an error, it prints a Fail verdict together with the the list of expected
output events to standard output and exits. The output of the commands given
to torx is written to standard output; diagnostics of the primer and adaptor
are printed to standard error.
To be added: the command line arguments
with which primer and adaptor are invoked.
A basic concept in torx is the
basic test action or event. As representation of an event torx basically
uses LOTOS-like notation (consisting of a gate name followed by zero or
more value-expressions, where each value-expression is preceded by an exclamation
mark (!)), with the extension that it may contain variables in the value-expressions.
These variables have the following form: var_type$nr where the $nr part
is optional. Here type is the type of the variable, and nr is a sequence
number to make variable occurences unique in the event. The special event
suspension is represented by (Suspension).
Each test step that torx executes
(as result of a command, or during execution in automatic mode) is reported
on standard output. Each test step appears on a separate line, containing
from left to right: the test step number, the iokind (input or output),
an opening parenthesis (() the channel name, a colon (:), the pco name,
a closing parenthesis ()), a colon (:), followed by the event. For suspension
events the pco name and preceding colon are omitted.
Apart from the
--help option, all torx options that can be set from the command line can
also be set as configuration file option (see torx-config(4)). The command
line options overrule settings specified in a configuration file.
The following
command line options are supported:
- --help
- print the version number and an
overview of the command line options, and exit.
- --batch
- start the tester in
batch mode. In this mode no adapter is started, but a batch test in Aldebaran
(.aut) format is generated on standard output, of depth as given with the
--depth flag. By default, this will have the format of a tree (i.e. there will
be no cycles) This is rather new and not very well tested!
- --batch-automaton
- generate the batch test in the form of an automaton (may contain cycles).
Note: this flag does not imply the --batch flag which must be separately
given.
- --depth nr
- (config: MAXSTEPS) start the tester in automatic mode, try
to execute nr of test events, and exit when done. The automatic execution
can be interrupted by giving a stop command, which causes a prompt to be
printed.
- --no-depth
- (config: MAXSTEPS) start torx in manual mode, with
unlimited depth (this is the default)
- --seed nr
- (config: SEED) use nr as
seed for random number generator(s)
- --no-seed
- (config: SEED) use random seed
(based on current time) (this is the default)
- --log file
- (config: LOGFILE)
write log to file. If file already exists torx will extend the file name
to be unique, by extending it with the string .~n~ where n is the smallest
number (from 0) that makes the file name unique.
- --no-log
- (config: LOGFILE)
do not write a log-file (this is the default)
- --logmon command
- (config: LOGMON)
start torx log monitoring command as background process that can continue
to run even after torx itself has exited, and write log to its standard
input. Multiple --logmon command line options may be given, to start multiple
commands (e.g. to use multiple viewers). The connection to command is made
via the utility program torx-logclient(1).
- --no-logmon
- (config: LOGMON) do not
start a log monitoring command (this is the default)
- --trace file
- (config:
TRACEFILE) read trace from file.
- --no-trace
- (config: TRACEFILE) do not use
a trace-file (this is the default)
The commands of torx are grouped
in three sections: 1) general ones, 2) commands that give information about
the current test execution status, without changing the current execution
`state', and 3) commands that execute a (sequence of) testing step(s).
- help
- print an overview of the recognized commands.
- quit
- (clean up and) exit the
tester
The following commands only print information, without
doing a test step. These commands do not cause a state change in primer
or adaptor.
- path
- print the path (the events done from start till now)
- menu
- print the menu (distinquishes inputs from outputs). Each menu-element is
printed on a separate line, containing from left to right: the iokind (input
or output), followed by a colon (:), the channel name between parentheses,
followed by the event.
- trace
- print the current event from the trace that
we follow
- state
- print the state(vector) (internal format of the primer)
- menusize
- print the size of the menu (this command will probably be deprecated)
- statesize
- print the size of the state(vector) (internal format) (this
command will probably be deprecated)
The following commands (may)
cause the execution of a testing step. Some commands can only be executed
if the (parenthesized) condition at the start of their explanation below
holds. If torx is unable to execute a command, it will print an error message
to standard output, and issue a new prompt. Note: in the commands next,
step, input, and output, the parentheses around the channel name argument
are part of the command syntax and can not be omitted.
- io
- select randomly
input or output, to be used in next step
- next (chan) input-event
- (input-event
isin menu) do one step using input-event
- next
- do one step (using the result
of last io command)
- step (chan) input-event
- synonym for next (chan) event
- step
- synonym for next
- input (chan) event
- (event isin menu) do one input
step using event
- input event
- (event isin menu) do one input step using
event
- input (chan)
- (menu of chan is non-empty) do one input step, from
channel chan
- input
- (input menu is non-empty) do one input step, from randomly
chosen channel
- output (chan)
- do one output step, from channel chan
- output
- do one output step, from randomly chosen channel
- auto
- switch to automatic
mode: do steps, randomly choosing in- and outputs until end of test, or
until interrupted by the stop command
- auto nr
- as auto, but do at most
nr steps
- usetrace
- use the current trace event for next step
- autotrace
- switch
to automatic mode: do steps, following the trace, until end of test, or
end of trace, or until interrupted by the stop command
- autotrace nr
- as
autotrace, but do at most nr steps
- stop
- interrupt the auto or autotrace
command and print a prompt.
Below we show a sh(1) shell script that
demonstrates how torx can be used in `batch' mode to repeatedly execute tests
upto a given number of test steps, for a given set of mutants, using a
different random number seed in each execution run. We assume here that
the mutant can be selected by setting the variable MUTANT in the environment.
The command used to invoke torx is split-up over several lines for clarity.
#!/bin/sh
first=1
beyond=1000
depth=1000000
mutants="111 222 333 444 555 666 777 888 999 000"
export MUTANT
i=$first
while test $i -lt $beyond
do
for m in $mutants
do
MUTANT=$m
torx --depth $depth \
--seed $i \
--log testloop.$i.$m.log \
config.if \
> testloop.$i.$m.out 2>&1
sleep 60
done
i=`expr $i + 1`
done
The assumption behind this script is that the implementation under test
will be started by (a shell script started by) torx (actually: by the torx
adaptor) which means that it (the implementation resp. the shell script)
will see variable MUTANT in its environment, and act on it.
torx-intro(1), mkprimer(1), torx-logclient(1), torx-primer(5), torx-adaptor(5),
torx-config(4), torx-log(4), xtorx(1), sh(1)
The diagnostic messages
of primer and adaptor are passed on to the standard error. If an error is
encountered in an command given to torx, an appropriate error message is
given (on standard output), and a new prompt is printed. The error messages
should be self-explanatory.
In case of normal termination (whether
or not an error is found) torx always returns with a 0 exit status. A non-zero
exit status will only be given when an (unforeseen?) internal or external
error makes normal termination impossible.
The diagnostics of primer
and adaptor appear interspersed with the output of torx; giving torx an
empty command (just press return) prints a new prompt.
Using a syntactically
wrong event as argument to a command will cause torx to exit.
The --depth
flag should be treated in a slightly different way: after automatically
doing the test steps required by this flag, torx should execute commands
given on standard input, until end-of-file on standard input, or until a
quit command is given. (however, this would require batch scripts to be
updated, to invoke torx with standard input redirected from /dev/null)
By Email: <torx_support@cs.utwente.nl>
This manual page documents
version 3.9.0 of torx.
Table of Contents
Prev
|
Table of Contents
|
Next
|
Appendix D: TorX Manual Pages: torx-sendpr(1) - problem report utility
|
|
Appendix D: TorX Manual Pages: torx_open(1) - run open/caesar tool on torx explorer program
|