mirror of
https://git.sakamoto.pl/laudom/http.sh.git
synced 2025-09-17 23:43:44 +02:00
169 lines
4.4 KiB
Markdown
169 lines
4.4 KiB
Markdown
# the test framework
|
|
|
|
We have a small test harness! It lives in `./tst.sh` in the root of the HTTP.sh repo. It's inspired
|
|
by some init systems, and a bit influenced by how APKBUILD/PKGBUILDs are structured. A very basic
|
|
test is attached below:
|
|
|
|
```
|
|
tst() {
|
|
return 0
|
|
}
|
|
```
|
|
|
|
A `tst()` function is all you need in a test. Running the test can be done like so:
|
|
|
|
```
|
|
$ ./tst.sh tests/example.sh
|
|
OK: tests/example.sh
|
|
|
|
|
|
Testing done!
|
|
OK: 1
|
|
FAIL: 0
|
|
```
|
|
|
|
If running multiple tests is desired, I recommend calling `./tst.sh tests/*`, and prepending the
|
|
filenames with numbers to make sure they run in the correct sequence.
|
|
|
|
You can also contain multiple tests in a file by grouping them into a function, and then adding the
|
|
function names to an array:
|
|
|
|
```
|
|
a() {
|
|
tst() {
|
|
return 0
|
|
}
|
|
}
|
|
b() {
|
|
tst() {
|
|
return 1
|
|
}
|
|
}
|
|
|
|
subtest_list=(
|
|
a
|
|
b
|
|
)
|
|
```
|
|
|
|
This will yield the following result *(output subject to change)*:
|
|
|
|
```
|
|
--- tests/example.sh ---
|
|
OK: a
|
|
FAIL: b
|
|
(res: )
|
|
|
|
|
|
Testing done!
|
|
OK: 1
|
|
FAIL: 1
|
|
```
|
|
|
|
Of note: `tst.sh` is designed in a way where *most* functions will fall through; If you'd like to
|
|
run the same test against a different set of checks (see below) then you *don't* need to redefine
|
|
the `tst()` function, just changing the checks is enough.
|
|
|
|
---
|
|
|
|
## return codes
|
|
|
|
The following return codes are defined:
|
|
|
|
- 0 as success
|
|
- 1 as error (test execution continues)
|
|
- 255 as fatal error (cleans up and exits immediately)
|
|
|
|
## determining success / failure
|
|
|
|
Besides very simple return-code based matching, `tst.sh` also supports stdout matching with the
|
|
following variables:
|
|
|
|
- `match` (matches the whole string)
|
|
- `match_sub` (matches a substring)
|
|
- `match_begin` (matches the beginning)
|
|
- `match_end` (matches the end)
|
|
- `match_not` (inverse substring match)
|
|
|
|
If any of those are defined, all except fatal return codes are ignored. If more than one of those
|
|
is defined, it checks the list above top-to-bottom and picks the first one that is set, ignoring
|
|
all others.
|
|
|
|
## special functions
|
|
|
|
The framework defines two special functions, plus a few callbacks that can be overriden:
|
|
|
|
### prepare
|
|
|
|
`prepare` runs **once** after definition, right before the test itself. As of now, it's the only
|
|
function that gets cleaned up after each run (by design; see section `statefullness` below)
|
|
|
|
By default (undefined state), `prepare` does nothing.
|
|
|
|
```
|
|
prepare() {
|
|
echo 'echo meow' > app/webroot/test.shs
|
|
}
|
|
|
|
tst() {
|
|
curl localhost:1337/test.shs
|
|
}
|
|
|
|
match="meow"
|
|
```
|
|
|
|
*(note: this test requires tst.sh to be used with http.sh, and for http.sh to be running)*
|
|
|
|
### cleanup
|
|
|
|
`cleanup` runs after every test. The name should be self-explanatory. Define as `cleanup() { :; }`
|
|
to disable behavior from previous tests.
|
|
|
|
By default (undefined state), `cleanup` does nothing.
|
|
|
|
```
|
|
prepare() {
|
|
echo 'echo meow' > app/webroot/test.shs
|
|
}
|
|
|
|
tst() {
|
|
curl localhost:1337/test.shs
|
|
}
|
|
|
|
cleanup() {
|
|
rm app/webroot/test.shs
|
|
}
|
|
|
|
match="meow"
|
|
```
|
|
|
|
*(note: same thing as above)*
|
|
|
|
### on_success, on_error, on_fatal
|
|
|
|
Called on every success, failure and fatal error. First two call `on_{success,error}_default`,
|
|
which increments the counter and outputs the OK/FAIL message. The third one just logs the FATAL,
|
|
cleans up and exits. Overloading `on_fatal` is not recommended; While overloading the other two,
|
|
make sure to add a call to the `_default` function, or handle the numbers gracefully by yourself.
|
|
|
|
## statefullness
|
|
|
|
This framework is designed in a way where a lot of the state is inherited from previous tests. This
|
|
is by-design, to make sure that there's less repetition in the tests themselves. It is up to the
|
|
author of the tests to remember about cleaning up variables and other state that could affect any
|
|
further tests in the chain.
|
|
|
|
Currently, state is cleaned up under the following circumstances:
|
|
- all `match` variables get cleaned up after every test
|
|
- `prepare()` function is reset after every test (so, each definition of `prepare` will run
|
|
exactly *once*)
|
|
- upon switching files, `tst()` and `cleanup()` get reset to initial values. Of note, those two
|
|
**do** get inherited between subtests in a single file!
|
|
- upon termination of the test harness, it tries to kill all child processes
|
|
|
|
The following state **is not** cleaned up:
|
|
- `tst()` and `cleanup()` between subtests in a single file
|
|
- `on_error()`, `on_success()` functions
|
|
- any global user-defined variables, also between files
|
|
- any started processes
|
|
- any modified files (we don't have a way to track those atm, although I may look into this)
|