rougail/docs/tutorial/dynfam.rst

.. _dynfam:

A dynamic family
================

.. objectives:: Objectives

   In this section we will learn how to create a dynamically built family.

   In a dynamically built family, instead of duplicating the definition of 
   identical variables in several families, they can be generated automatically.

.. prerequisites:: Prerequisites

    - We assume that Rougail's library is :ref:`installed <installation>` on your computer.

    - It is possible to retrieve the current state of the various Rougail files manipulated in this tutorial step
      by checking out the corresponding tag of the `rougail-tutorials` git repository.
      Each tag corresponds to a stage of progress in the tutorial.
      Of course, you can also decide to copy/paste or download the tutorial files contents while following the tutorial steps.

    If you want to follow this tutorial with the help of the corresponding :tutorial:`rougail-tutorials git repository <src/branch/1.1>`, 
    this workshop page corresponds to the tags :tutorial:`v1.1_060 <src/tag/v1.1_060>` to :tutorial:`v1.1_061 <src/tag/v1.1_061>` 
    in the repository.

    ::
   
       git clone https://forge.cloud.silique.fr/stove/rougail-tutorials.git
       git switch --detach v1.1_060

We handled the HTTPS mode in the previous section. But there's more modes to handle.
Let's turn back to the firefox's configuration page:

.. image:: images/soksv5.png

We see that we need to handle the SOCKS configuration in addition to the HTTPS configuration.
Moreover, we can see that these two groups of variables are similar in the structure: 
they both have a host and a port.

There are two proxies that are to be configured : 

- the HTTPS proxy
- the SOCKS proxy

As they have the same structure, would it be possible to define the two of them 
in one shot?

.. note:: It's not the place here to describe what the HTTP and SOCKS protocols are. 
          The interesting point here is that they are very similar in our firefox's 
          configuration and that we can do batch processing.

.. index:: dynamic family

Family: a dynamic family
----------------------------

With Rougail, it is possible to create some kind of a model of family. 
Kind of a generic family declaration. 
We call this generic family creation process a dynamic creation because as we will see below, 
these families exist at the very moment we define their **identifiers**.

First, here is what we need to make (without identifiers):

.. code-block:: yaml

   https_proxy:
     description: HTTPS Proxy
     ...

     address:
       description: HTTPS address
       ...

     port:
       description: HTTPS Port
       ...

   sock_proxy:
     description: SOCKS Proxy
     ...

     address:
       description: SOCKS address
       ...

     port:
       description: SOCKS Port
       ...

Now with identifiers, we have the ability to declare our families this way:

.. code-block:: yaml

  "{{ identifier }}_proxy":
    description: "{{ identifier }} Proxy"
    dynamic:
      - HTTPS
      - SOCKS

    address:
      description: "{{ identifier }} address"

    port:
      description: "{{ identifier }} port"

.. index:: identifier

.. type-along:: What is exactly an identifier?

If you used the YAML declaration tool named `Ansible <https://docs.ansible.com>`_, 
the variable used to iterate over multiple values in a task is called an **`item`**.
We call it an identifier.

It is a symbol used in the context of a loop. For example:

.. code-block:: yaml
   :caption: A code example of an ansible loop
   
   - name: Loop example with 'item'
     ansible.builtin.debug:
       msg: "The current value is {{ item }}"
     loop:
       - value1
       - value2
       - value3

This code will output: 

.. code-block:: text

   The current value is value1
   The current value is value2
   The current value is value3

In the Rougail context, we name this item an identifier because it is an item 
that allow us to define dynamically family names.

.. glossary::

   identifier

      In the :ref:`dynamic family creation field <dynfam>` we call an identifier
      an item that defines a family name. An item is a variable on which an iteration 
      on keywords will be carried out. 

      An :term:`identifier` is a local variable, used only for creating multiple 
      iterations, used for creating multiple families in only one declaration.

      It allows us to declare very similar families in a more generic way.

Here is the syntax we are using that allows the declaration of multiple families at one time:

.. code-block:: yaml

  "{{ identifier }}_proxy":
    description: "{{ identifier }} Proxy"
    dynamic:
      - HTTPS
      - SOCKS

This identifier is a parameter that enables us to create two families named `https_proxy` and `socks_proxy`:

.. code-block:: yaml

  https_proxy:
    description: "HTTPS Proxy"

  socks_proxy:
    description: "SOCKS Proxy"

.. attention:: Be careful when choosing your identifiers items: pay attention that the family 
               that will be dynamically created has not been declared before in some other 
               YAML structure file.

               If you define a dynamic family with the `https` item that will 
               build a `https_proxy` family and if this familiy already exists,
               then rougail will raise a family/variable override warning.
               
               When choosing a name, 
               
               - rougail will put it in lowercase
               - only ASCII and underscore ("`_`") characters are allowed.

Here is our dynamic familiy in situation in the :file:`firefox/20-manual.yml` structure file.

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_060/firefox/20-manual.yml
   :language: yaml 
   :caption: The :file:`firefox/20-manual.yml` structure file with the dynamic families declaration
   
..
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          variable: _.use_for_https
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port
    ...

When we launch the Rougail CLI, we can see the concrete families 
and variables that have been created:

Here is the user data file on which we will launch the Rougail CLI:

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/tag/v1.1_060/config/01/config.yml
   :language: yaml
   :caption: The :file:`config/01/config.yml` user data file

..
    ---
    proxy_mode: Manual proxy configuration
    manual:
      http_proxy:
        address: http.proxy.net
        port: 3128
      use_for_https: false
      https_proxy:
        address: https.proxy.net

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/tag/v1.1_060/config/01/cmd_ro.txt
   :class: terminal 
   
.. 
    rougail -m firefox/ -u yaml -yf config/01/config.yml


The Rougail CLI outputs this: 

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/tag/v1.1_060/config/01/output_ro.html
   :class: output

.. 
    rougail -m firefox/proxy.yml -u yaml --yaml.filename config/proxy.yml
    ╭─────────────────── Caption ────────────────────╮
    │ Variable              Default value            │
    │ Unmodifiable variable Modified value           │
    │                       (Original default value) │
    ╰────────────────────────────────────────────────╯
    Variables:
    ┗━━ 📂 Manual proxy configuration
        ┣━━ 📂 HTTP Proxy
        ┃   ┣━━ 📓 HTTP address: ... (loaded from the YAML file "userdata/proxy.yml")
        ┃   ┗━━ 📓 HTTP Port: ... (8080 - loaded from the YAML file "userdata/proxy.yml")
        ┣━━ 📓 Also use this proxy for HTTPS: true
        ┣━━ 📂 HTTPS Proxy
        ┃   ┣━━ 📓 HTTPS address: ...
        ┃   ┗━━ 📓 HTTPS port: ...
        ┗━━ 📂 SOCKS Proxy
            ┣━━ 📓 SOCKS address: ...
            ┗━━ 📓 SOCKS port: ...

We can see that the dynamic families have been created: 

- an `HTTPS Proxy` family
- a `SOCKS Proxy` family

and, as we expected, containing an address and a port.

A conditional disabled variable with dynamic identifier
--------------------------------------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the `v1.1_061` version::
     
          git switch --detach v1.1_061

Here is the final version of the HTTPS and SOCKS structure file: 

We have added:

- a conditional hidden family property
- a default value

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/tag/v1.1_061/firefox/20-manual.yml
   :language: yaml
   :caption: The final :file:`firefox/20-proxy.yml` structure file

.. 
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          variable: _.use_for_https
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: HTTPS
    ...

The disabled `property` is assigned here to the `version` variable 
in the case where the identifier is HTTPS.

.. 
    version:
      description: SOCKS host version used by proxy
      choices:
        - v4
        - v5
      default: v5
      disabled:
        type: identifier
        when: HTTPS

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/tag/v1.1_061/config/01/config.yml
   :language: yaml
   :caption: The :file:`config/01/config.yml` user data file

..
    ---
    proxy_mode: Manual proxy configuration
    manual:
      http_proxy:
        address: http.proxy.net
        port: 3128
      use_for_https: false
      https_proxy:
        address: https.proxy.net

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/tag/v1.1_061/config/01/cmd_ro.txt
   :class: terminal 
   
.. 
    rougail -m firefox/ -u yaml -yf config/01/config.yml

The Rougail CLI outputs this: 

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/tag/v1.1_061/config/01/output_ro.html
   :class: output

..
    Variables:
    ┣━━ 📓 Configure Proxy Access to the Internet: Manual proxy configuration ◀ loaded from the YAML file "config/01/config.yml" (⏳ No proxy)
    ┗━━ 📂 Manual proxy configuration
        ┣━━ 📂 HTTP Proxy
        ┃   ┣━━ 📓 HTTP address: http.proxy.net ◀ loaded from the YAML file "config/01/config.yml"
        ┃   ┗━━ 📓 HTTP Port: 3128 ◀ loaded from the YAML file "config/01/config.yml" (⏳ 8080)
        ┣━━ 📓 Also use this proxy for HTTPS: false ◀ loaded from the YAML file "config/01/config.yml" (⏳ true)
        ┣━━ 📂 HTTPS Proxy
        ┃   ┣━━ 📓 HTTPS address: https.proxy.net ◀ loaded from the YAML file "config/01/config.yml" (⏳ http.proxy.net)
        ┃   ┗━━ 📓 HTTPS port: 3128
        ┗━━ 📂 SOCKS Proxy
            ┣━━ 📓 SOCKS address: http.proxy.net
            ┣━━ 📓 SOCKS port: 3128
            ┗━━ 📓 SOCKS host version used by proxy: v5

.. keypoints:: Key points

   - We now know how to build a dynamic family is, with setting its identifier.
   - we have a new use case for the `disabled` property. 
     We know how to disable conditionaly a dynamic family.