Newer
Older
# KDTesting
Python library for testing applications using IO.
# Test
Test is container that holds name, arbitrary and value provided by said arbitrary (if you use constructor with None, it will generate value using arbitrary). The test also contains functions `shrink` and `to_str` (both just use the functions from arbitrary on the value held by the test).
Tester is abstract class, the most important function `run_test(test: Test[G])` is not defined. Tester defines how shrinking works etc.
Tester that runs tests on your implementation and the reference implementation, then runs your 'process output tests' on the outputs. More below.
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
## ProcessOutputTest
Type is `Callable[[ProcessOutput, ProcessOutput], bool]`, it takes the outputs (in format provided by the `subprocess.run`) and returns bool.
There are some presets:
- `stdout_equality()`: returns test, that only compares the raw stdout
- `stdout_processed_equality(test_preprocessor, exp_preprocessor)`: returns test that compares processed stdouts. `test_preprocessor` and `exp_preprocessor` are `Callable[[str], str]` that "process" the stdout (you have to define these, there is `str_id` that returns the input unchaged `x -*> x`)
## Example use
```python
def ignore_numbers(x: str):
return re.sub(r'\d+', '', x)
# `program_tst` and `program_exp` are absolute paths pointing to the executables
# 3 - verbosity (default is 5, too much)
tester = RefImplTester(program_tst, program_exp, 3)
tester.add_output_test(stdout_processed_eqality(ignore_numbers, ignore_numbers))
for i in range(5):
# `command_sequence` is a custom arbitrary that returns sequence of
# `\n` delimitered commands
tester.add_test(Test(f"all_{i}", command_sequence))
# runs the test, returns (reduced test, number of shrinks) tuple
reduced_tests = tester.run()
# prints the failed tests, first the name, the number of shrinks, the test,
# then saves it as .tst file
for test, shrinks in reduced_tests:
print(test.name, shrinks)
print(test.to_str())
tester.save_shrunk(reduced_tests)
# also saves the original tests
tester.save_tests()
```
# Arbitraries
`Arbitrary[T]` defines an object capable of providing arbitrary value with type `T`, shrinking value of type `T` to a simpler instance and converting value of type `T` to `str`.
To define your custom Arbitrary, either extend one of the predefined structures (below) or extend Arbitrary. Note that you have to define functions `shrink`, you don't have to define `to_str` but it will probably create undesirable output. The function `get` uses the value generator from the constructor.
The constructor of `Arbitrary[T]` takes ValueGenerator, function with type `Callable[[], T]`, this function is used in get(). There are some predefined value generatos, see ValueGenerator section.
shrinks any value (created by the Arbitrary) to simpler value
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
### To string
`to_str(value: T) -> str`
converts the value to string
## The arbitraries currently present are:
### NumberA(min_value, max_value)
get returns: number in [min_value, max_value)
shrinks value to: val // 2, val moved towards 0
### ConstantA(constant)
get returns: constant
shrinks value to: nothing
### TupleA(arbs)
get returns: tuple where i-th value uses i-th arbitrary from arbs
shrinks value to: list of different tuples with shrunk values
### ListA(arb, min_size, max_size)
get returns: list with length [min_size, max_size)
shrinks value to: shorter lists
### ChoiceA(choices)
get returns: tuple (index of chosen arb, value generated by that arb), choice is random
shrinks value to: tuples with shrunk values (index of the chosen arb, value shrunk using said arb)
# ValueGenerator
There are some predefined factories. Note that you have to CALL the factory, to recieve value generator.
*- = returns generator that creates
## predefined value generators creators
### Atomic
- `integer_gen(min_value: int, max_value: int)` *- random integer in range [min_value, max_value)
- `pos_integer_gen(max_value: int)` *- random integer in range [1, max_valaue)
- `lower_case_let_gen()` *- lower case letter
- `upper_case_let_gen()` *- upper case letter
### Generic
- `constant_gen(const: T)` *- the constant
- `oneof_gen(constants: List[T])` *- one of the constants
- `vector_gen(gen: Callable[[], T], size: int)` *- list with constant size with elements generated by the generator
- `listof_gen(gen: Callable[[], T], min_size: int, max_size: int)` *- list with random size [min_value, max_value) with elements generated by the generator
- `tupleof_gen(generators: List[ValueGenerator])` *- tuple created from the values of the different generators
- `choices_gen(choices: List[Tuple[int, Callable[[], T]]])` *- choices are list of tuples in form (weight, generator), generates value created by one of the generators, chooses using weighted random