documentation

This commit is contained in:
egarette@silique.fr 2022-12-21 12:14:49 +01:00
parent e1a447da7d
commit 5bec5dffed
39 changed files with 1947 additions and 660 deletions

View file

@ -17,6 +17,10 @@ Clone projects:
- https://cloud.silique.fr/gitea/risotto/rougail
- https://cloud.silique.fr/gitea/risotto/risotto
## Documentation
[Documentation](doc/README.md)
## Set up
Set up Risotto:
@ -31,7 +35,7 @@ In risotto.conf change the dataset directory.
Set up infrasctructure:
```bash
cp server.json.example server.json
cp server.yml.example server.yml
```
Modify infrastructure description as required.
@ -45,37 +49,5 @@ Generate the configuration:
Send configuration to remote server:
```bash
HOST=cloud.silique.fr
rm -f installations.tar
tar -cf installations.tar installations
scp installations.tar root@$HOST:
```
## Deploy
In host:
```bash
cd
rm -rf installations
tar xf installations.tar
cd installations
```
Set up host:
```bash
./install_host cloud.silique.fr
```
Build container image:
```bash
./install_images cloud.silique.fr
```
Set up the containers and start them up:
```bash
./install_machines cloud.silique.fr
ansible-playbook -i ansible/inventory.py ansible/playbook.yml
```

View file

@ -2,4 +2,33 @@
# Risotto
![Schéma](schema.png "Schéma")
## A dataset
- [Dataset example](dataset_example/dataset.md)
## Infrastructure
- [Infrastructure](infrastructure.md)
- [Examples](dataset_example/infrastructure.md)
## risotto.conf
```toml
[directories]
datasets = ['<path_to_dataset_base>/seed']
dest = 'installations'
dest_templates = 'templates'
[cert_authority]
email = '<email>'
country = 'FR'
locality = 'Dijon'
state = 'France'
org_name = 'Silique'
org_unit_name = 'Cloud'
```
## Usage
![Schema](schema.png "Schéma")

View file

@ -0,0 +1,87 @@
# Risotto dataset simple examples
This tutorial aims to show how create a dataset to deploy a [Caddy](https://caddyserver.com/) server via Risotto.
Attention it has no other virtues than to be educational. It is not intended for production use.
See [Rougail documentation for more details about dictionaries, templates and patches](https://cloud.silique.fr/gitea/risotto/rougail/src/branch/main/doc/README.md).
The project can be divided into three application services:
- caddy-common: an application service containing the information common to the two other application services
- caddy-https: a standalone http/https server
- caddy-https-rp: a https only server served behind a reverse proxy
## caddy-common
Start by creating the project tree:
```
seed/caddy-common/
├── dictionaries
├── templates
└── manual
   └── image
   └── preinstall
```
Then describe the application service in [seed/caddy-common/applicationservice.yml](seed/caddy-common/applicationservice.yml).
Also a dictionary [seed/caddy-common/dictionaries/20-caddy.yml](seed/caddy-common/dictionaries/20-caddy.yml) with
- the activation of the caddy service in the "multi-user" target. This service needs some templates:
- the main configuration's [/etc/caddy/Caddyfile](seed/caddy-common/templates/Caddyfile) to include other /etc/caddy/Caddyfile.d/\*.caddyfile
- /etc/caddy/Caddyfile.d/risotto.caddyfile with appropriate configuration (this file is not part of this application service)
- a [sysusers](https://www.freedesktop.org/software/systemd/man/sysusers.d.html) file [/sysusers.d/0caddy.conf](seed/caddy-common/templates/sysuser-caddy.conf) to create the system user "caddy"
- a [tmpfiles](https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html) file [/tmpfiles.d/0caddy.conf](seed/caddy-common/templates/tmpfile-caddy.conf) to create the directory "caddy_root_directory" and volatile directory "/var/lib/caddy"
- a family "caddy" (Caddy web server) with a filename variable "caddy_root_directory" (The root path of the site) with default value "/srv/caddy".
Finally, create a script to build the image with the caddy package: [seed/caddy-common/manual/image/preinstall/caddy.sh](seed/caddy-common/manual/image/preinstall/caddy.sh).
## caddy-https
Start by creating the project tree:
```
seed/caddy-https-rp/
├── dictionaries
└── templates
```
Then describe the application service in [seed/caddy-https/applicationservice.yml](seed/caddy-https/applicationservice.yml) with OS and caddy-common dependencies.
Also create a dictionary [seed/caddy-https/dictionaries/25-caddy.yml](seed/caddy-https/dictionaries/25-caddy.yml) to define the variables:
- caddy_domain: the domain where Caddy should listen to
- caddy_ca_file, caddy_crt_file and caddy_key_file: certificat for this domain
- redefine the variable incoming_ports to open the ports 80 and 443
And new templates:
- [seed/caddy-https/templates/risotto.caddyfile](seed/caddy-https/templates/risotto.caddyfile)
- [seed/caddy-https/templates/ca_HTTP.crt](seed/caddy-https/templates/ca_HTTP.crt)
- [seed/caddy-https/templates/caddy.key](seed/caddy-https/templates/caddy.key)
- [seed/caddy-https/templates/caddy.crt](seed/caddy-https/templates/caddy.crt)
## caddy-https-rp
Start by creating the project tree:
```
seed/caddy-https-rp/
├── dictionaries
├── patches
└── templates
```
Then describe the application service in [seed/caddy-https-rp/applicationservice.yml](seed/caddy-https-rp/applicationservice.yml) with OS, caddy-common and reverse-proxy-client dependencies.
By default, reverse proxy certificate is only readable by "root" user. In the dictionary [seed/caddy-https-rp/dictionaries/25-caddy.yml](seed/caddy-https-rp/dictionaries/25-caddy.yml) we change the user to "caddy".
And add Caddy configuration's file [seed/caddy-https-rp/templates/risotto.caddyfile](seed/caddy-https-rp/templates/risotto.caddyfile).
This template use mainly variables defined in reverse-proxy application service.
Finally add a patch to modify Caddyfile to not starts Caddy in port 80: [seed/caddy-https-rp/patches/Caddyfile.patch](seed/caddy-https-rp/patches/Caddyfile.patch).
Patches should only use if a template file is define in an other dataset. You should instead add a condition in the template. But for educational reasons we made a patch in this case.

View file

@ -0,0 +1,38 @@
# Examples
## Caddy as HTTPS server
The [servers.yml](servers.caddy-https.yml):
- we create only the zone "external"
- we create a module "caddy"
- we define an host "host.example.net":
- servers are containerized with [machined](https://freedesktop.org/wiki/Software/systemd/machined/), so service application is "host-systemd-machined"
- the provide application service is "provider-systemd-machined"
- we define a server "caddy"
## Caddy behind a Nginx reverse proxy
The [servers.yml](servers.caddy-https-rp.yml):
- we create the zone "external" and a zone "revprox" between "revprox" and "caddy" servers
- we create three module:
- "revprox": the reverse proxy (with "letsencrypt" application service if needed)
- "nsd": to manage local DNS name
- "caddy"
- we define an host "host.example.net":
- servers are containerized with [machined](https://freedesktop.org/wiki/Software/systemd/machined/), so service application is "host-systemd-machined"
- the provide application service is "provider-systemd-machined"
- we define servers:
- revprox in zones "external" and "revprox"
- nsd in zone "revprox"
- caddy in zone "revprox"
You must add a index.html file in "/var/lib/risotto/srv/caddy.in.example.net/caddy/".

View file

@ -0,0 +1,2 @@
format: '0.1'
description: Caddy's common files

View file

@ -0,0 +1,25 @@
services:
- service:
- name: caddy
target: multi-user
file:
- text: /etc/caddy/Caddyfile
engine: 'none'
- text: /etc/caddy/Caddyfile.d/risotto.caddyfile
- text: /sysusers.d/0caddy.conf
source: sysuser-caddy.conf
engine: 'none'
- text: /tmpfiles.d/0caddy.conf
source: tmpfile-caddy.conf
engine: 'none'
variables:
- family:
- name: caddy
description: Caddy web server
variables:
- variable:
- name: caddy_root_directory
type: filename
description: The root path of the site
value:
- text: /srv/caddy

View file

@ -0,0 +1 @@
PKG="$PKG caddy"

View file

@ -0,0 +1,43 @@
# The Caddyfile is an easy way to configure your Caddy web server.
#
# https://caddyserver.com/docs/caddyfile
#>GNUNUX
# Global options
{
# remove administration tool
admin off
}
#<GNUNUX
# The configuration below serves a welcome page over HTTP on port 80. To use
# your own domain name with automatic HTTPS, ensure your A/AAAA DNS record is
# pointing to this machine's public IP, then replace `http://` with your domain
# name. Refer to the documentation for full instructions on the address
# specification.
#
# https://caddyserver.com/docs/caddyfile/concepts#addresses
#GNUNUX http:// {
# Set this path to your site's directory.
#GNUNUX root * /usr/share/caddy
# Enable the static file server.
#GNUNUX file_server
# Another common task is to set up a reverse proxy:
# reverse_proxy localhost:8080
# Or serve a PHP site through php-fpm:
# php_fastcgi localhost:9000
# Refer to the directive documentation for more options.
# https://caddyserver.com/docs/caddyfile/directives
#GNUNUX}
# As an alternative to editing the above site block, you can add your own site
# block files in the Caddyfile.d directory, and they will be included as long
# as they use the .caddyfile extension.
import Caddyfile.d/*.caddyfile

View file

@ -0,0 +1,2 @@
g caddy 998 -
u caddy 998:998 "Caddy web server" /var/lib/caddy /sbin/nologin

View file

@ -0,0 +1,2 @@
d /var/lib/caddy 750 caddy caddy - -
d %%caddy_root_directory 750 root caddy - -

View file

@ -0,0 +1,6 @@
format: '0.1'
description: Caddy
depends:
- base-fedora-36
- reverse-proxy-client
- caddy-common

View file

@ -0,0 +1,9 @@
variables:
- family:
- name: revprox
variables:
- variable:
- name: revprox_client_cert_owner
redefine: true
value:
- text: caddy

View file

@ -0,0 +1 @@
PKG="$PKG caddy"

View file

@ -0,0 +1,11 @@
--- a/Caddyfile 2022-12-21 11:51:32.834081202 +0100
+++ b/Caddyfile 2022-12-21 11:51:26.354030537 +0100
@@ -7,6 +7,8 @@
{
# remove administration tool
admin off
+ # do not start caddy on port 80
+ auto_https disable_redirects
}
#<GNUNUX

View file

@ -0,0 +1,20 @@
# listen to all reverse proxy domains
%for %%domain in %%revprox_client_external_domainnames
https://%%domain {
# import reverse proxy certificate
# do not try to check zerossl and let's encrypt file
tls %%revprox_client_cert_file %%revprox_client_key_file {
ca_root %%revprox_client_ca_file
}
# log to the console
log {
output stdout
format console
level info
}
# root directory
root * %%caddy_root_directory
# it's a file server
file_server
}
%end for

View file

@ -0,0 +1,2 @@
g caddy 998 -
u caddy 998:998 "Caddy web server" /var/lib/caddy /sbin/nologin

View file

@ -0,0 +1,2 @@
d /srv/caddy 750 root caddy - -
d /var/lib/caddy 750 caddy caddy - -

View file

@ -0,0 +1,5 @@
format: '0.1'
description: Caddy as standalone HTTPs serveur
depends:
- base-fedora-36
- caddy-common

View file

@ -0,0 +1,72 @@
services:
- service:
- name: caddy
file:
- file_type: variable
text: caddy_ca_file
source: ca_HTTP.crt
- file_type: variable
text: caddy_crt_file
source: caddy.crt
- file_type: variable
text: caddy_key_file
source: caddy.key
variables:
- family:
- name: network
variables:
- variable:
- name: incoming_ports
redefine: true
value:
- text: 80
- text: 443
- name: caddy
variables:
- variable:
- name: caddy_domain
type: domainname
description: Domain name
- name: caddy_ca_file
type: filename
description: Caddy CA filename
hidden: true
- name: caddy_key_file
type: filename
description: Caddy private key filename
hidden: true
- name: caddy_crt_file
type: filename
description: Caddy public key filename
hidden: true
constraints:
- fill:
- name: calc_value
param:
- type: variable
text: tls_ca_directory
- text: ca_HTTP.crt
- name: join
text: /
target:
- text: caddy_ca_file
- fill:
- name: calc_value
param:
- type: variable
text: tls_cert_directory
- text: caddy.crt
- name: join
text: /
target:
- text: caddy_crt_file
- fill:
- name: calc_value
param:
- type: variable
text: tls_key_directory
- text: caddy.key
- name: join
text: /
target:
- text: caddy_key_file

View file

@ -0,0 +1 @@
PKG="$PKG caddy"

View file

@ -0,0 +1,57 @@
# The Caddyfile is an easy way to configure your Caddy web server.
#
# https://caddyserver.com/docs/caddyfile
# The configuration below serves a welcome page over HTTP on port 80. To use
# your own domain name with automatic HTTPS, ensure your A/AAAA DNS record is
# pointing to this machine's public IP, then replace `http://` with your domain
# name. Refer to the documentation for full instructions on the address
# specification.
#
# https://caddyserver.com/docs/caddyfile/concepts#addresses
#>GNUNUX
#http:// {
#listen only in https
{
admin off
}
%for %%domain in %%revprox_client_external_domainnames
https://%%domain {
tls %%revprox_client_cert_file %%revprox_client_key_file {
ca_root %%revprox_client_ca_file
}
log {
output stdout
format console
level info
}
#<GNUNUX
# Set this path to your site's directory.
#>GNUNUX
# root * /usr/share/caddy
root * /srv/caddy
#<GNUNUX
# Enable the static file server.
file_server
# Another common task is to set up a reverse proxy:
# reverse_proxy localhost:8080
# Or serve a PHP site through php-fpm:
# php_fastcgi localhost:9000
# Refer to the directive documentation for more options.
# https://caddyserver.com/docs/caddyfile/directives
}
%end for
# As an alternative to editing the above site block, you can add your own site
# block files in the Caddyfile.d directory, and they will be included as long
# as they use the .caddyfile extension.
#GNUNUX import Caddyfile.d/*.caddyfile

View file

@ -0,0 +1 @@
%%get_chain(cn=%%caddy_domain, authority_cn=%%caddy_domain, authority_name="HTTP", hide=%%hide_secret)

View file

@ -0,0 +1 @@
%%get_certificate(%%caddy_domain, 'HTTP', type="server", hide=%%hide_secret)

View file

@ -0,0 +1 @@
%%get_private_key(cn=%%caddy_domain, authority_name='HTTP', type="server", hide=%%hide_secret)

View file

@ -0,0 +1,18 @@
# listen to all reverse proxy domains
https://%%caddy_domain {
# use certificate
# do not try to check zerossl and let's encrypt file
tls %%caddy_crt_file %%caddy_key_file {
ca_root %%caddy_ca_file
}
# log to the console
log {
output stdout
format console
level info
}
# root directory
root * %%caddy_root_directory
# it's a file server
file_server
}

View file

@ -0,0 +1,2 @@
g caddy 998 -
u caddy 998:998 "Caddy web server" /var/lib/caddy /sbin/nologin

View file

@ -0,0 +1,2 @@
d /srv/caddy 750 root caddy - -
d /var/lib/caddy 750 caddy caddy - -

View file

@ -0,0 +1,48 @@
zones:
external:
network: 192.168.45.0/24
host_ip: 192.168.45.1
start_ip: 192.168.45.10
domain_name: in.example.net
revprox:
network: 192.168.46.0/24
host_ip: 192.168.46.1
start_ip: 192.168.46.10
domain_name: revprox.in.example.net
modules:
revprox:
- nginx-reverse-proxy
- letsencrypt
nsd:
- nsd
caddy:
- caddy-https-rp
hosts:
host.example.net:
applicationservices:
- host-systemd-machined
applicationservice_provider: provider-systemd-machined
values:
general.network.interfaces.interface_names:
- ens3
general.network.output_interface: ens3
servers:
nsd:
module: nsd
informations:
zones_name:
- revprox
revprox:
module: revprox
informations:
zones_name:
- external
- revprox
caddy:
module: caddy
informations:
zones_name:
- revprox
values:
general.revprox.revprox_client.revprox_client_external_domainnames:
- caddy.example.net

View file

@ -0,0 +1,26 @@
zones:
external:
network: 192.168.45.0/24
host_ip: 192.168.45.1
start_ip: 192.168.45.10
domain_name: in.example.net
modules:
caddy:
- caddy-https
hosts:
host.example.net:
applicationservices:
- host-systemd-machined
applicationservice_provider: provider-systemd-machined
values:
general.network.interfaces.interface_names:
- ens3
general.network.output_interface: ens3
servers:
caddy:
module: caddy
informations:
zones_name:
- external
values:
general.caddy.caddy_domain: caddy.example.net

41
doc/infrastructure.md Normal file
View file

@ -0,0 +1,41 @@
# Infrastructure
The infrastructure is define in a uniq YAML file: servers.yml:
## Zones
The idea:
- separate the networks according to the uses
- there is no route to each other
Ideally only one area has an Internet access.
Internet access is, in fact, firewall rules.
This network is usually called "external".
The other networks are only there for the communication between server and client.
The host must have an IP in this network.
IP inside this network are deliver automaticly.
A network is call a "zone".
## Modules
A module is simply a list of application services. An system image is build with informations define in application service.
## Hosts
A host is a server on which container or VM are running.
Define the host means define:
- application services to configure the host and VM
- application service provider to define the provider to apply on each VM
- values to adapt the configuration
- servers, the list of VM with :
- the corresponding module
- informations (like zone)
- values
Host must only be a Debian 11 (Bullseye) from now.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 138 KiB

After

Width:  |  Height:  |  Size: 323 KiB

File diff suppressed because it is too large Load diff

Before

Width:  |  Height:  |  Size: 139 KiB

After

Width:  |  Height:  |  Size: 167 KiB

BIN
logo.png

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.3 KiB

After

Width:  |  Height:  |  Size: 8.4 KiB

View file

@ -2,13 +2,16 @@
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg
width="37.29702mm"
height="38.628922mm"
viewBox="0 0 37.297019 38.628922"
width="37.297001mm"
height="38.629002mm"
viewBox="0 0 37.297 38.629002"
version="1.1"
id="svg5"
inkscape:version="1.1.2 (0a00cf5339, 2022-02-04)"
inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
sodipodi:docname="logo.svg"
inkscape:export-filename="logo.png"
inkscape:export-xdpi="149.26"
inkscape:export-ydpi="149.26"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns="http://www.w3.org/2000/svg"
@ -23,15 +26,17 @@
inkscape:pagecheckerboard="0"
inkscape:document-units="mm"
showgrid="false"
inkscape:zoom="1.1011145"
inkscape:cx="63.571954"
inkscape:cy="79.010857"
inkscape:window-width="1033"
inkscape:window-height="1063"
inkscape:window-x="26"
inkscape:window-y="23"
inkscape:window-maximized="0"
inkscape:current-layer="layer1" />
inkscape:zoom="4.404458"
inkscape:cx="63.685475"
inkscape:cy="75.378174"
inkscape:window-width="1920"
inkscape:window-height="1011"
inkscape:window-x="0"
inkscape:window-y="0"
inkscape:window-maximized="1"
inkscape:current-layer="layer1"
inkscape:showpageshadow="2"
inkscape:deskcolor="#d1d1d1" />
<defs
id="defs2" />
<g
@ -40,10 +45,10 @@
id="layer1"
transform="translate(-75.0784,-36.897831)">
<rect
style="fill:#ffffff;fill-rule:evenodd;stroke-width:1.5;stroke-linecap:square;paint-order:fill markers stroke;fill-opacity:1"
id="rect848"
width="37.29702"
height="38.628918"
style="fill:#f6f7d7;fill-opacity:1;fill-rule:evenodd;stroke:none;stroke-width:2.04884;stroke-linecap:square;paint-order:fill markers stroke"
id="rect8118"
width="37.297001"
height="38.629002"
x="75.0784"
y="36.897831" />
<rect
@ -54,12 +59,19 @@
x="75.0784"
y="36.897831" />
<rect
style="fill:#008700;fill-opacity:1;fill-rule:evenodd;stroke:#008700;stroke-width:1.35088;stroke-linecap:square;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;paint-order:fill markers stroke"
style="fill:#008700;fill-opacity:1;fill-rule:evenodd;stroke:#008700;stroke-width:1.84143;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;paint-order:fill markers stroke"
id="rect18918"
width="16.625383"
height="6.7146907"
x="85.29847"
y="65.215286" />
width="29.788723"
height="6.963315"
x="78.625404"
y="40.178349" />
<rect
style="fill:#008700;fill-opacity:1;fill-rule:evenodd;stroke:#008700;stroke-width:1.84143;stroke-linecap:square;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;paint-order:fill markers stroke"
id="rect18918-5"
width="29.788723"
height="6.963315"
x="78.625114"
y="65.0494" />
<text
xml:space="preserve"
style="font-size:10.5833px;line-height:1.15;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#4d4d4d;stroke-width:0.265;stroke-miterlimit:4;stroke-dasharray:none"
@ -68,13 +80,36 @@
id="text2080"><tspan
sodipodi:role="line"
id="tspan2078"
style="font-weight:bold;text-align:center;text-anchor:middle;fill:#008700;fill-opacity:1;stroke-width:0.265;stroke-miterlimit:4;stroke-dasharray:none"
style="font-weight:bold;text-align:center;text-anchor:middle;fill:#f6f7d7;fill-opacity:1;stroke-width:0.265;stroke-miterlimit:4;stroke-dasharray:none"
x="93.5"
y="47.586319">RIS</tspan><tspan
sodipodi:role="line"
style="font-weight:bold;text-align:center;text-anchor:middle;fill:#4d4d4d;stroke-width:0.265;stroke-miterlimit:4;stroke-dasharray:none"
x="93.5"
y="59.757114"
id="tspan2082">OTTO</tspan></text>
id="tspan2082">OTTO</tspan><tspan
sodipodi:role="line"
style="font-weight:bold;text-align:center;text-anchor:middle;fill:#f6f7d7;fill-opacity:1;stroke-width:0.265;stroke-miterlimit:4;stroke-dasharray:none"
x="93.5"
y="71.92791"
id="tspan7995" /></text>
<circle
style="fill:#008700;fill-opacity:1;fill-rule:evenodd;stroke:#f6f7d7;stroke-width:0.56;stroke-linecap:square;stroke-dasharray:none;stroke-opacity:1;paint-order:fill markers stroke"
id="path19218"
cx="103.00674"
cy="68.43734"
r="1.7277808" />
<path
style="fill:#f6f7d7;fill-opacity:1;stroke:#f6f7d7;stroke-width:0.56;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1"
d="M 82.1984,66.707831 H 95.287674"
id="path19357" />
<path
style="fill:#f6f7d7;fill-opacity:1;stroke:#f6f7d7;stroke-width:0.6;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1"
d="M 82.1984,70.167831 H 95.287664"
id="path19357-6" />
<path
style="fill:#f6f7d7;fill-opacity:1;stroke:#f6f7d7;stroke-width:0.56;stroke-linecap:round;stroke-linejoin:round;stroke-dasharray:none;stroke-opacity:1"
d="M 82.1984,68.45114 H 95.287664"
id="path19357-5" />
</g>
</svg>

Before

Width:  |  Height:  |  Size: 3 KiB

After

Width:  |  Height:  |  Size: 4.8 KiB