Some ideas about writing test case titles and organising them into a Test Case Space.
I had a dream about "Test Space".
My final year project was "The State Space representation of rectifier Systems" and it took me a term to understand the title.
So what is a "space"? You have a list of dimensions. Set each dimension to a value, and this defines a position in the space.
A system has inputs and outputs. The outputs depend of the inputs and history. The history used could be a number of items and not just a simple one dimensional number, hence "space".
So for Test Case Space Analysis, you can write down lots of lists, each one could be a dimension in the Test Case Space.
A simple test sequence could be run at many points in the Testcase Space, the position defined by the configuration.
We are told to write down a list of test cases with steps and then asked What percentage have you done?
"Test Case Space" is like "State Space" but for testing.
So each test has a position in the Test Case Space which has a very rich range of dimensions.
A telephone service has an A-leg , a B-leg and a C-leg, and different call flows depending on number dialled.
The A-leg call can be made from one of a big list of device configurations. This list could be thought of as a dimension. The B leg could be another list of device configurations so another dimension.
The dialled number can also determin a call flow, another list and dimension.
The B-leg can be a number of different outcomes, like busy, or answered by a voice device or answering machine or a modem.
The number of pass / fail results can get very big, very rapidly.
My computer has a Part number made up of many fields. So a simple Model name may be appended by a string of letters to represent CPU speed, Disk size, and many other configurations terms.
Test cases are similar and can be titled similar to a part number.
ICs have complex part numbers as an IC can have a range of package types, speeds.
Integer numbers use positional notation so the number 123 is 1x100 + 2x10 + 3x1
Pre decimalization, the cost of an item was made of of pounds , shillings and pence.
The positional notation used multiple bases. N^P where Nis not constant and P is the position.
price = pounds + shillings + pence , where there are 12 pence to a shilling and 20 shillings to a pound.
You can code the tests using a similar idea.
Some test dimensions are not integers, e.g. make a call and hold the call open for a finite time, the time is a real number, but may be quantized.
Sorting could be done like a number. You may want to report test results based on different parameters.
Some computer languages use positional parameters in functional calls, some allow lists of optional parameter_name=value;
Sorting of test cases may need to be ordered by different parameters depending on what needs to be reported.
So the Test Case Space could have a very large number of test cases, which may be an infinite number of cases.
A test case may record a list of measurements, or repeated to get each measurement.
This all gets very complicated.
I used a protocol tester that allowed you to send a message.
A way to simplify the test was to measure a live call flow and then reproduce it.
Have a function that initialzed the test message.
InitIAM(); // a default message based on a typical message.
Then for each test case modifiy the message for each test case.
This makes it clear what is being modified. e.g.
![[Home]](../ipstats.gif)
InitIamCLI( p1, p2, p3 ){
InitIAM();
IAM.cli.x = p1;
IAM.pn.x = p2;
IAM.gi.x = p3;
}
Then call SendMessage() and cope with the call flow.
It may be that the test cases are randomly picked and recorded by automation analysis of test logs.
The tests could be plotted on a Test Case Space and tests added where there is a lack of results.
http://www.dougrice.plus.com/hp/Theory/Index.htm has this image.
For years I have been trying to work out how to list tests.
Using simple nested for next loops quickly became daunting and the test document became big.
The project managers say how many tests and what percentage have you done?
By using a random number, to pick a position in the Test Space.
Maybe use multiple random numbers, for different dimensions.
Fuzz Testing is a technique using a "bad cop" then "good cop" tests and see which one breaks the system.
"Monti carlo" testing was a term used to simulate multiple Telephone connections and try and understand an envelope of results.
Dig a trench and see what you find. Have a list of asserts that you consider good behaviour to provide pass or fail.
Our testers captured multiple call flows. It was possible to dump the calls as text files.
I worked out I could sorted the text files and count unique lines.
Could I report how messages used parameters?
I prefixed lines of a message with the message name and line number from start of the message.
Then I could sort and count unique lines.
I could measure how many times a parameter had a particular value.
This provided an automated confidence check.
My computer uses a file directory, which uses a tree structure. You could think of each folder as a dimensions.
When I started work, my organisation used a dot notatation to describe the position in the tree.
I worked in TA13.3.2 which described a node on the organisational tree.
So the total number tests could be itterate through the list of tree nodes using a recursive search.
It gets big - too big. You only have a day to test, before the project has moved on.
Some thoughts, Using a random number to pick where to start, seems a good way.
