truh/http.sh
1
0
Fork 0
mirror of https://git.sakamoto.pl/laudom/http.sh.git synced 2025-08-11 16:03:33 +02:00
Code Issues Projects Releases Wiki Activity

Compare commits

base: truh:9ad46350afde76f3e7108035b97d219323af821d
Branches Tags
truh:master
truh:rust
truh:pack
truh:v0.97.2
truh:v0.97.1
truh:v0.97
..
compare: truh:f2d72ef6eed3d8acc483d46872284c2df202797b
Branches Tags
truh:master
truh:rust
truh:pack
truh:v0.97.2
truh:v0.97.1
truh:v0.97

8 commits
9ad46350af ... f2d72ef6ee

Author SHA1 Message Date
sdomi
f2d72ef6ee tests: fixup invalid header test 2025-04-06 06:29:39 +02:00
sdomi
c6d3dfa045 server: fixup hangs on empty POST reqs 2025-04-06 06:18:52 +02:00
sdomi
d5aaa1c265 template: tpl includes don't need to prepend the expr anymore 2025-04-06 05:05:57 +02:00
sdomi
f9e1be8a90 template: horrible hack implementing iterators in included templates 2025-04-06 04:39:01 +02:00
sdomi
6a9ec3bf71 template: small style/performance fixes 2025-04-06 03:56:33 +02:00
sdomi
e9f5ab52d2 docs: remove some ambiguity 2025-04-06 01:51:27 +02:00
sdomi
f889062633 docs: mention template include's path quirk 2025-04-05 23:53:03 +02:00
sdomi
3080e38cad docs: document the template engine 2025-04-05 23:35:27 +02:00
6 changed files with 394 additions and 36 deletions
Download patch file Download diff file Expand all files Collapse all files

1
README.md
View file

@ -13,6 +13,7 @@ We have some guides and general documentation in the [docs](docs/) directory. Am
- [CLI usage](docs/running.md) - [CLI usage](docs/running.md)
- [Tests](docs/tests.md) - [Tests](docs/tests.md)
- [HTTP Router](docs/router.md) - [HTTP Router](docs/router.md)
- [Template engine](docs/template.md)
- [List of security fixes](docs/sec-fixes/) - [List of security fixes](docs/sec-fixes/)
## Dependencies ## Dependencies

193
docs/template-examples.md Normal file
View file

@ -0,0 +1,193 @@
# HTTP.sh: template usage examples
## Basic example
Create a new .shs file with the following contents:
```
#!/usr/bin/env bash
declare -A str
str[title]="Hello, world!"
str[test]="meow"
render str "${cfg[namespace]}/templates/main.htm"
```
`render` is the core of the templating engine; it takes an assoc array, iterates over it, applies
additional magic and outputs the response directly to stdout. It is likely the final thing you want
to run in your script.
The script above has referenced an HTML file; For this example, we put it under
`app/templates/main.htm`, but you're free to use any directory structure for this.
```
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>{{.title}}</title>
</head>
<body>
{{.test}}
</body>
</html>
```
![netscape 3.06 gold screenshot, showing our awesome page](https://f.sakamoto.pl/IwILCemig.png)
## Boolean if statements
Following is an example script which simulates a coin toss:
```
#!/usr/bin/env bash
declare -A str
str[title]="Coin flip!"
if (( RANDOM%2 == 0 )); then
str[?random]=_
fi
render str "${cfg[namespace]}/templates/main.htm"
```
And the corresponding template:
```
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>{{.title}}</title>
</head>
<body>
{{start ?random}}
It's heads!
{{else ?random}}
It's tails!
{{end ?random}}
</body>
</html>
```
![another netscape screenshot. the page is titled Coin flip! and it shows that it rolled heads](https://f.sakamoto.pl/IwIQT0d6w.png)
50% of the time the variable will be set, 50% it won't. Hence, it will display either heads or tails :)
Of note: if you hate repeating yourself, this template can be done inline:
```
It's {{start ?random}}heads{{else ?random}}tails{{end ?random}}!
```
The effect is exactly the same. This is quite useful for adding CSS classes.
## Loop example
This API is pending a rewrite due to how convoluted it is.
```
#!/usr/bin/env bash
declare -A str
str[title]="foreach example"
nested_declare list # "array of arrays"
declare -A elem # temporary element
for i in {1..32}; do
elem[item]="$i" # assign $i to the temporary element
nested_add list elem # add elem to list; this creates a copy you can't modify
done
# once we have a full list of elements, assign it to the array passed to render
str[_list]=list
render str "${cfg[namespace]}/templates/main.htm"
```
And the template...
```
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>{{.title}}</title>
</head>
<body>
{{start _list}}
{{.item}}<br>
{{end _list}}
</body>
</html>
```
The result repeats the whole "subtemplate" between list start and end:
![list so long that the numbers go off-screen!](https://f.sakamoto.pl/IwI0slukA.png)
This is very useful for rendering data in tables:
```
<table>
<tr>
<th>number</th>
</tr>
{{start _list}}
<tr>
<td>{{.item}}</td>
<td>whatever...</td>
</tr>
{{end _list}}
</table>
```
![our example, now rendered as a table](https://f.sakamoto.pl/IwIf39cYw.png)
### integration with notORM
notORM's `data_iter` function works great with nested_add; Body of a callback function can be
treated as equal to a for loop:
```
declare -A elem
nested_declare list
x() {
elem[ns]="${data[2]}"
elem[domain]="${data[1]}"
nested_add list elem
}
data_iter storage/zones.dat "$username" x
str[title]="SERVFAIL :: zone list"
str[_list]=list
```
## date pretty-printing
```
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>{{.title}}</title>
</head>
<body>
Current time is {{+time}}
</body>
</html>
```
```
#!/usr/bin/env bash
declare -A str
str[title]="time pretty-print"
str[+time]="$EPOCHSECONDS"
render str "${cfg[namespace]}/templates/main.htm"
```
![netscape displays the current date and time](https://f.sakamoto.pl/IwIvf3Axw.png)
If you get quirky with the `<meta http-equiv="refresh" content="1">`, you can even make it
auto update! (don't)
![oh gosh it refreshes now. whyyyyyyyyyyyyyyyy](https://f.sakamoto.pl/simplescreenrecorder-2025-04-05_22.57.36.png)

157
docs/template.md Normal file
View file

@ -0,0 +1,157 @@
# HTTP.sh: template engine
We have a basic template engine! It's somewhat limited in capabilities compared to engines you might
have previously used, but we're working on making it better :3
Note: the `templates` subdirectory in the HTTPsh repo is entirely unrelated to the template engine,
and it will be removed in a future release. Please ignore it.
For practical examples, see the [template examples](template-examples.md) page.
## Tag schema
- Tags always start with `{{` and end with `}}`.
- Tags can't include whitespace, outside of special iter/boolean tags defined below
- Tag identifiers can contain letters, numbers, dashes and underscores (`[a-zA-Z0-9_-]`).
Other characters may work but are NOT RECOMMENDED.
- Tag identifiers are always prefixed by the tag type. This is also reflected in the code, outside
of simple replaces which MUST skip the dot in the array assignment.
- Identifiers are represented by `<name>` later in this document.
## API
`render <assoc_array> <file> [recurse]`
The first param points to an associative array containing the replacement data. Second one points
to a file containing the template itself. Third is optional, and controls whether `render` will
recurse or not. This is mostly used internally, you likely won't ever need to set it.
## Simple replace
| | |
| --- | --- |
| In the template | `{{.<name>}}` |
| In the code | `array[<name>]="<value>"` |
| Notes | For your convenience, code representation skips the dot. |
**Important**: to simplify your life (and protect your application), simple replaces ALWAYS use
html_encode behind the scenes. This means that you're safe to assign any value to them without
prior sanitization.
## Raw replace
| | |
| --- | --- |
| In the template | `{{@<name>}}` |
| In the code | `array[@<name>]="<value>"` |
Same as a simple replace, but doesn't do html_encode. Useful if you want to guarantee unmangled
output (for filling out hidden form values, etc.)
## Template includes
| | |
| --- | --- |
| In the template | `{{#<path>}}` |
| In the code | n/a |
Template includes are special, in that you don't have to define them in the array.
They get processed first to "glue together" one singular template.
Currently, the path starts at the root of HTTPsh's directory. We don't support expanding variables
inside the include tag, so for now you'll need to hardcode `{{#app/templates/...}}`. This will
likely get changed in a future release, starting the path in your namespace.
**Warning**: No recursion is supported within included templates; This means that you can't have
an "include chain". Furthermore, some interactions between included templates and loops/ifs are
a bit wonky; This will get ironed out at some point (sorry!)
## Boolean if statements
| | |
| --- | --- |
| In the template | `{{start ?<name>}} ... {{end ?<name>}}` |
| In the template (alt.) | `{{start ?<name>}} ... {{else ?<name>}} ... {{end ?<name>}}` |
| In the code | `array[?<name>]=_` |
| Notes | Can be used both inline and not. See [examples page](template-examples.md) for more details. |
**Important**: Currently, you can't have two checks for the same variable. If needed, set a second
variable in the code and check for that. Fix TBD.
This is a *boolean operator*. The only supported mode of operation is checking whether
a variable is set or not.
## Loops
| | |
| --- | --- |
| In the template | `{{start _<name>}} ... {{end _<name>}}` |
| In the code | `array[_name]="<reference>"` |
Each loop extracts the area between start/end markers, and executes another `render` internally.
You have to provide it with an "array of arrays", essentially an intermediate holding references.
This is usually done through `nested_declare <array>` and `nested_add <array> <temporary>`.
Essentially, this boils down to:
```
nested_declare list # "array of arrays"
declare -A elem # temporary element
for i in {1..32}; do
elem[item]="$i" # assign $i to the temporary element
nested_add list elem # add elem to list; this creates a copy you can't modify
done
# once we have a full list of elements, assign it to the array passed to render
str[_list]=list
```
A more detailed usage description is available on the [template examples](template-examples.md) page.
### Leaky temporary array
You should excercise caution when handling the temporary arrays; Calling `unset elem` on the end
of each loop may be a good idea if you can't guarantee that all of your elements will always have
values. Otherwise, values from previous iterations may leak to the current one, potentially causing confusion.
## Loop indexes
| | |
| --- | --- |
| In the template | `{{-index}}` |
| In the code | n/a |
| Notes | Doesn't resolve at all outside loops. Counter starts at 0 and gets incremented with every element. |
## Date pretty-printing
| | |
| --- | --- |
| In the template | `{{+<name>}}` |
| In the code | `array[+<name>]="<timestamp>"` |
This saves you from a few messy calls to `date`. Input is a UNIX timestamp.
The date format can be overriden by changing a config variable. Default is
`cfg[template_date_format]='%Y-%m-%d %H:%M:%S'`.
## URI slices
| | |
| --- | --- |
| In the template | `{{-uri-<level>}}` |
| In the code | n/a |
| Notes | Level must be a number. URI is always terminated with a slash, even if your last object is a file. |
Takes the current URI path, and slices it using `/` as a delimeter, going from the left.
Given an URL `http://localhost:1337/hello/world/asdf`...
- `{{-uri-0}}` -> `/`
- `{{-uri-1}}` -> `/hello/`
- `{{-uri-2}}` -> `/hello/world/`
- `{{-uri-3}}` -> `/hello/world/asdf/`
- `{{-uri-4}}` -> none (higher values are always empty)
This is very useful when creating menus; Instead of relying on hardcoded values, if the page is always
on *the same URI level*, one can create links such as `<a href="{{-uri-2}}meow">(...)</a>`, which will always
resolve to the same file; This eliminates a whole class of bugs where trailing slashes would break some
poorly-written relative URLs.

2
src/server.sh
View file

@ -252,12 +252,14 @@ if [[ "${r[post]}" == true ]] && [[ "${r[status]}" == 200 || "${r[status]}" ==
if [[ "${r[content_length]}" ]]; then if [[ "${r[content_length]}" ]]; then
read -r -N "${r[content_length]}" data read -r -N "${r[content_length]}" data
else else
if read -t0; then
data= data=
while read -r line; do while read -r line; do
data+="$line" data+="$line"
done done
unset line unset line
fi fi
fi
if [[ "${r[payload_type]}" == "urlencoded" ]]; then if [[ "${r[payload_type]}" == "urlencoded" ]]; then
unset IFS unset IFS

65
src/template.sh
View file

@ -13,13 +13,43 @@ function render() {
local template="$(tr -d "${_tpl_ctrl}" < "$2" | sed -E 's/\\/\\\\/g')" local template="$(tr -d "${_tpl_ctrl}" < "$2" | sed -E 's/\\/\\\\/g')"
fi fi
local buf= local buf=
local garbage=
local -n ref=$1 local -n ref=$1
# process file includes;
# recursion is currently unsupported here, i feel like it may break things?
if [[ "$template" == *'{{#'* && "$3" != true ]]; then
local subtemplate=
while read key; do
# below check prevents the loop loading itself as a template.
# this is possibly not enough to prevent all recursions, but
# i see it as a last-ditch measure. so it'll do here.
if [[ "$file" == "$2" ]]; then
subtemplate+="s${_tpl_ctrl}\{\{\#$key\}\}${_tpl_ctrl}I cowardly refuse to endlessly recurse\!${_tpl_ctrl}g;"
elif [[ -f "$key" ]]; then
local input="$(tr -d "${_tpl_ctrl}${_tpl_newline}" < "$key" | sed 's/\&/<2F>UwU<77>/g')"
garbage+="$input"$'\n'
input="$(tr $'\n' "${_tpl_newline}" <<< "$input")" # for another hack
subtemplate+="s${_tpl_ctrl}\{\{\#$key\}\}${_tpl_ctrl}${input}${_tpl_ctrl};"
_template_find_special_uri "$(cat "$key")"
fi
done <<< "$(grep -Poh '{{#\K(.*?)(?=}})' <<< "$template")"
buf+="${subtemplate}"
fi
local key local key
IFS=$'\n' IFS=$'\n'
for key in ${!ref[@]}; do for key in ${!ref[@]}; do
if [[ "$key" == "_"* ]]; then # iter mode if [[ "$key" == "_"* ]]; then # iter mode
local subtemplate="$(grep "{{start $key}}" -A99999 <<< "$template" | grep "{{end $key}}" -B99999 | tr '\n' "${_tpl_newline}")" # THE MOST EVIL OF ALL HACKS:
# we're scraping a subtemplate from our main template.
# HOWEVER: this fails on included templates, because they're not real.
# this means that iterators can't work on included templates
#
# workaround? collect all includes, concatenate them all together and just.
# use that pile of garbage here along with the real template. it works!
local subtemplate="$(grep "{{start $key}}" -A99999 <<< "$template"$'\n'"$garbage" | grep "{{end $key}}" -B99999 | tr '\n' "${_tpl_newline}")"
local -n asdf=${ref["$key"]} local -n asdf=${ref["$key"]}
local j local j
local value='' local value=''
@ -31,7 +61,7 @@ function render() {
(( _index++ )) (( _index++ ))
done done
buf+="s${_tpl_ctrl}\{\{start $key\}\}.*\{\{end $key\}\}${_tpl_ctrl}\{\{$key\}\}${_tpl_ctrl};s${_tpl_ctrl}\{\{$key\}\}${_tpl_ctrl}$(tr -d "${_tpl_ctrl}" <<< "$value" | sed -E "s${_tpl_ctrl}"'\{\{start '"$key"'\}\}'"${_tpl_ctrl}${_tpl_ctrl};s${_tpl_ctrl}"'\{\{end '"$key"'\}\}'"${_tpl_ctrl}${_tpl_ctrl}")${_tpl_ctrl};" buf+="s${_tpl_ctrl}\{\{start $key\}\}.*\{\{end $key\}\}${_tpl_ctrl}\{\{$key\}\}${_tpl_ctrl};s${_tpl_ctrl}\{\{$key\}\}${_tpl_ctrl}$(tr -d "${_tpl_ctrl}" <<< "$value" | sed "s${_tpl_ctrl}{{start $key}}${_tpl_ctrl}${_tpl_ctrl};s${_tpl_ctrl}{{end $key}}${_tpl_ctrl}${_tpl_ctrl}")${_tpl_ctrl};"
unset "$subtemplate" unset "$subtemplate"
elif [[ "$key" == "@"* && "${ref["$key"]}" != '' ]]; then elif [[ "$key" == "@"* && "${ref["$key"]}" != '' ]]; then
local value="$(tr -d "${_tpl_ctrl}${_tpl_newline}" <<< "${ref["$key"]}" | sed -E 's/\&/<2F>UwU<77>/g')" local value="$(tr -d "${_tpl_ctrl}${_tpl_newline}" <<< "${ref["$key"]}" | sed -E 's/\&/<2F>UwU<77>/g')"
@ -51,12 +81,7 @@ function render() {
elif [[ "$key" == '?'* ]]; then elif [[ "$key" == '?'* ]]; then
local _key="\\?${key/?/}" local _key="\\?${key/?/}"
# TODO: check if this is needed? buf+="s${_tpl_ctrl}"'\{\{start '"$_key"'\}\}((.*)\{\{else '"$_key"'\}\}.*\{\{end '"$_key"'\}\}|(.*)\{\{end '"$_key"'\}\})'"${_tpl_ctrl}"'\2\3'"${_tpl_ctrl};"
# the code below makes sure to resolve the conditional blocks
# *before* anything else. I can't think of *why* this is needed
# right now, but I definitely had a reason in this. Question is, what reason.
buf+="s${_tpl_ctrl}"'\{\{start '"$_key"'\}\}((.*)\{\{else '"$_key"'\}\}.*\{\{end '"$_key"'\}\}|(.*)\{\{end '"$_key"'\}\})'"${_tpl_ctrl}"'\2\3'"${_tpl_ctrl};${buf}" # MAYBE_SLOW
elif [[ "${ref["$key"]}" != "" ]]; then elif [[ "${ref["$key"]}" != "" ]]; then
if [[ "$3" != true ]]; then if [[ "$3" != true ]]; then
@ -71,32 +96,12 @@ function render() {
done done
unset IFS unset IFS
# process file includes;
# achtung: even though this is *after* the main loop, it actually executes sed reaplces *before* it;
# recursion is currently unsupported here, i feel like it may break things?
if [[ "$template" == *'{{#'* && "$3" != true ]]; then
local subtemplate=
while read key; do
# below check prevents the loop loading itself as a template.
# this is possibly not enough to prevent all recursions, but
# i see it as a last-ditch measure. so it'll do here.
if [[ "$file" == "$2" ]]; then
subtemplate+="s${_tpl_ctrl}\{\{\#$key\}\}${_tpl_ctrl}I cowardly refuse to endlessly recurse\!${_tpl_ctrl}g;"
elif [[ -f "$key" ]]; then
subtemplate+="s${_tpl_ctrl}\{\{\#$key\}\}${_tpl_ctrl}$(tr -d "${_tpl_ctrl}${_tpl_newline}" < "$key" | tr $'\n' "${_tpl_newline}" | sed 's/\&/<2F>UwU<77>/g')${_tpl_ctrl};"
_template_find_special_uri "$(cat "$key")"
fi
done <<< "$(grep -Poh '{{#.*?}}' <<< "$template" | sed 's/{{#//;s/}}$//')"
buf="${subtemplate}$buf"
fi
_template_find_special_uri "$template" _template_find_special_uri "$template"
buf+="$(_template_gen_special_uri)" buf+="$(_template_gen_special_uri)"
if [[ "$3" != true ]]; then # are we recursing? if [[ "$3" != true ]]; then # are we recursing?
tr '\n' ${_tpl_newline} <<< "$template" | sed -E -f <( tr '\n' "${_tpl_newline}" <<< "$template" | sed -E -f <(
tr '\n' "${_tpl_newline}" <<< "$buf" | sed -E $'s/\02;\01/\02;/g;s/\02g;\01/\02g;/g' # i'm sorry what is this sed replace?? tr '\n' "${_tpl_newline}" <<< "$buf" | sed $'s/\02;\01/\02;/g;s/\02g;\01/\02g;/g' # i'm sorry what is this sed replace??
echo -n 's/\{\{start \?([a-zA-Z0-9_-]*[^}])\}\}(.*\{\{else \?\1\}\}(.*)\{\{end \?\1\}\}|.*\{\{end \?\1\}\})/\3/g' echo -n 's/\{\{start \?([a-zA-Z0-9_-]*[^}])\}\}(.*\{\{else \?\1\}\}(.*)\{\{end \?\1\}\}|.*\{\{end \?\1\}\})/\3/g'
) | tr "${_tpl_newline}" '\n' | sed -E 's/<2F>UwU<77>/\&/g' ) | tr "${_tpl_newline}" '\n' | sed -E 's/<2F>UwU<77>/\&/g'
else else

2
tests/01-http-basic.sh
View file

@ -158,7 +158,7 @@ server_req_header_dup() {
server_req_header_invalid() { server_req_header_invalid() {
tst() { tst() {
# we have to trick curl into sending an invalid header for us # we have to trick curl into sending an invalid header for us
curl -s "localhost:1337/meow.shs" -H $'a:\nasdf asdf asdf asdf' -H "meow: asdf" curl -s "localhost:1337/meow.shs" -H $'meow:\nasdf asdf asdf asdf' -H "a: aaaa"
} }
match_not="asdf" match_not="asdf"