rougail/docs/tutorial/jinja.rst

Playing with Jinja 
====================

.. objectives:: Objectives

   In this section we will learn how to create new ways of calculation.

   Up to now, our only way of dynamically (that is, during the runtime) calculating 
   a value is to point on another variable's value. But this is not the only way. 

   We will learn how to insert `Jinja templating language <https://jinja.palletsprojects.com/en/stable/>`_ into our structure files.
   Please note that we do not intend to template our YAML files. 
   That is a completely different activity. 
   We will simply insert YAML code to calculate the value of a variable, a property, or a parameter.

.. prerequisites:: Prerequisites

    - We assume that Rougail's library is :ref:`installed <installation>` on your computer.

    - It is possible to retrieve the current state of the various Rougail files manipulated in this tutorial step
      by checking out the corresponding tag of the `rougail-tutorials` git repository.
      Each tag corresponds to a stage of progress in the tutorial.
      Of course, you can also decide to copy/paste or download the tutorial files contents while following the tutorial steps.

    If you want to follow this tutorial with the help of the corresponding :tutorial:`rougail-tutorials git repository <src/branch/1.1>`, 
    this workshop page corresponds to the tags :tutorial:`v1.1_070 <src/tag/v1.1_070>` to :tutorial:`v1.1_072 <src/tag/v1.1_072>` 
    in the repository.

    ::
   
       git clone https://forge.cloud.silique.fr/stove/rougail-tutorials.git
       git switch --detach v1.1_070

.. type-along:: Why the Jinja templating engine ?
 
We are going to embed some `Hinja templating code <https://jinja.palletsprojects.com/en/stable/>`_ in our structure file.

.. glossary::

    Jinja

        `Jinja <https://jinja.palletsprojects.com>`_ is a template engine.
        we are using Jinja in a classical way, that is, Jinja allows us to handle different cases,
        for example with the `if` statement.

 
.. questions:: What about a `Jinja` calculation?

The :term:`Jinja` templating language enables some complex calculation. 
For example we can code more complex behavior for hiding or disabling a variable, that is 
not only depending on the value of another variable as we saw before.

A conditional hidden family with Jinja
---------------------------------------

Here we are going to use the Jinja conditional (testing) statement.

Here is our structure file:

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/firefox/20-manual.yml
   :language: yaml 
   :caption: The :file:`firefox/20-manual.yml` structure file with some `jinja` code in the `hidden` property

..
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          jinja: |-
            {% if _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: HTTPS
    ...

In this Jinja code, we are testing the `use_for_https` variable:

.. code-block:: jinja

     {% if _.use_for_https %}
         HTTPS is same has HTTP
     {% endif %}

If this variable is set to `true`, the `"HTTPS is same has HTTP"` expression 
will appear.

.. note:: Have a look at the `Jinja website's test section <https://jinja.palletsprojects.com/en/stable/templates/#tests>`_
          for a closer look about testing a variable.

But first, let's set our `use_for_https` variable to `false`: 

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/config.yml
   :language: yaml 
   :caption: The :file:`config/01/config.yml` user data file with the `use_for_https` set to `false`

..
    ---
    proxy_mode: Manual proxy configuration
    manual:
      http_proxy:
        address: http.proxy.net
        port: 3128
      use_for_https: false
      https_proxy:
        address: https.proxy.net
   
Nothing special appears when we launch the Rougail CLI:

.. raw:: html 
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/cmd_ro.txt
   :class: terminal
   
..
    rougail -m firefox/ -u yaml -yf config/01/config.yml

We have this output: 

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/output_ro.html
   :class: output

.. 
    Variables:
    ┣━━ 📓 Configure Proxy Access to the Internet: Manual proxy configuration ◀ loaded from the YAML file "config/01/config.yml" (⏳ No proxy)
    ┗━━ 📂 Manual proxy configuration
        ┣━━ 📂 HTTP Proxy
        ┃   ┣━━ 📓 HTTP address: http.proxy.net ◀ loaded from the YAML file "config/01/config.yml"
        ┃   ┗━━ 📓 HTTP Port: 3128 ◀ loaded from the YAML file "config/01/config.yml" (⏳ 8080)
        ┣━━ 📓 Also use this proxy for HTTPS: false ◀ loaded from the YAML file "config/01/config.yml" (⏳ true)
        ┣━━ 📂 HTTPS Proxy
        ┃   ┣━━ 📓 HTTPS address: https.proxy.net ◀ loaded from the YAML file "config/01/config.yml" (⏳ http.proxy.net)
        ┃   ┗━━ 📓 HTTPS port: 3128
        ┗━━ 📂 SOCKS Proxy
            ┣━━ 📓 SOCKS address: http.proxy.net
            ┣━━ 📓 SOCKS port: 3128
            ┗━━ 📓 SOCKS host version used by proxy: v5

If we set the `use_for_https` to `true` in the :file:`config.yml` user data file, 
then we launch again the Rougail CLI:

.. raw:: html 
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/cmd_ro.txt
   :class: terminal

then we have a warning:

::

    🔔 Warning
    ┗━━ Manual proxy configuration
        ┗━━ HTTPS Proxy
            ┗━━ HTTPS address: 🔔 family "HTTPS Proxy" has property hidden, 
            so cannot access to "HTTPS address", 
            it will be ignored when loading from the YAML file "config/01/config.yml"

What is interesting here is to launch the Rougail CLI in the :term:`read write mode`:

.. code-block:: bash
   :class: terminal 

    rougail -m firefox/ --cli.root manual.https_proxy -u yaml \
            -yf config/01/config.yml --cli.read_write

Then we have an error:

.. code-block:: bash

    ERROR: cannot access to optiondescription "HTTPS Proxy" 
    because has property "hidden" 
    (HTTPS is same has HTTP)

.. note:: Note that we can see the "HTTPS is same has HTTP" string (the one that was present in the Jinja code) 
          as an explanation inside the error message.

.. type-along:: Without calculation

Let's reason on the previous HTTPS proxy configuration's manual mode settings:

.. code-block:: yaml

      use_for_https:
        default: true

      https_proxy:
        type: family
        description: HTTPS Proxy
        hidden:
          variable: _.use_for_https

We see here that the `https_proxy` family is hidden 
depending on the value of another variable. This is what we saw before, 
how to hide a variable.

Now we can code the same behavior in a somehow different way: 

.. code-block:: yaml

      https_proxy':
        description: HTTPS Proxy
        hidden:
          jinja: |-
            {% if _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}

This code has the same result and yes, it's done in a more complicated way.
We have replaced this simple `hidden` property variable parameter by 
a Jinja parameter. The hidden process is done by a calculation. 

The fact is that it has same result, but it opens more possibilities,
we will see them later.

Jinja with a description
-----------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the `v1.1_071` version::
     
          git switch --detach v1.1_071

It is preferable to include a description related to the Jinja calcuation. 
That allows the calculation performed by the Jinja code to be understood rapidely 
without having to read the code itself. 
This is useful for the :term:`integrators <integrator>` who review these :term:`structure files <structure file>`.

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_071/firefox/20-manual.yml
   :language: yaml 
   :caption: The :file:`firefox/20-manual.yml` structure file with a description added to the `Jinja` code

..
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          jinja: |-
            {% if _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}
          description: in HTTPS case if "_.use_for_https" is set to "true"
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: HTTPS
    ...

This description will appear `in the documentation output for the family <https://forge.cloud.silique.fr/stove/rougail-tutorials/src/tag/v1.1_071#https-proxy-or-socks-proxy>`_

With this rougail command:

.. code-block:: bash

    rougail -m firefox -o doc --doc.output_format html > readme.html

It outputs an HTML report, here is a part of it about the `https_proxy` with the `hidden` property:

.. raw:: html
   :class: output 
       
    manual.<i>https</i>_proxy <br/>

    <b>Hidden</b>: in HTTPS case if "manual.use_for_https" is set to "true" <br/>

    <b>Identifiers</b>: 
    <ul><li>HTTPS</li>

We can see that the hidden explanation for the `manual.https_proxy` family 
is the `hidden` property's description.

Jinja with a parameter
---------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the `v1.1_072` version::
     
          git switch --detach v1.1_072

Regarding dynamic families as in the use case we are dealing with, here is an interesting Rougail feature:
it is possible to retrieve the family identifiers and declare them as parameters in the Jinja calculation.

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_072/firefox/20-manual.yml
   :language: yaml 
   :caption: The :file:`firefox/20-manual.yml` structure file with an identifier type parameter in the hidden property

.. 
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          jinja: |-
            {% if my_identifier == 'HTTPS' and _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}
          description: in HTTPS case if "_.use_for_https" is set to "true"
          params:
            my_identifier:
              type: identifier
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: HTTPS
    ...

We have added an `identifier` type parameter to the `hidden` property:

.. code-block:: yaml

    hidden:
      params:
        my_identifier:
          type: identifier

Thus we can refine our Jinja code and hide the `https_proxy` dynamic family,
that is only in the case where the identifier is `HTTPS`.

.. code-block:: yaml

    hidden:
      jinja: |-
        {% if my_identifier == 'HTTPS' and _.use_for_https %}
          HTTPS is same has HTTP
        {% endif %}

In this Jinja code, we can understand that only the `https_proxy` dynamic family 
will be hidden when the `_.use_for_https` variable is set to `true`.

Let's set the `_.use_for_https` variable to `true`.
We have a warning:

.. code-block:: text

    🔔 Warning
    ┗━━ Manual proxy configuration
        ┗━━ HTTPS Proxy
            ┗━━ HTTPS address: 🔔 family "HTTPS Proxy" has property hidden, 
                so cannot access to "HTTPS address", it will be ignored 
                when loading from the YAML file "config/01/config.yml"

It's helpful to run this command in read/write mode: 

.. code-block:: bash
   :class: terminal 

   rougail -m firefox/ --cli.root manual.https_proxy -u yaml \
           -yf config/01/config.yml --cli.read_write

because at that moment we get an error:

.. code-block:: bash
   :class: output

   ERROR: cannot access to optiondescription "HTTPS Proxy" 
   because has property "hidden" (HTTPS is same has HTTP)

So the Jinja code worked well since the `https_proxy` family is hidden only when 
the `use_for_https` variable is set to `true` and the identifier is HTTPS 
(the `socks_proxy` family, however, is not hidden).

.. keypoints:: Key points

   We have seen that the `disabled` or `hidden` properties could be calculated.
   The default values can be calculated too. We'll see a bunch of examples later on.

   **progress**
      
   - calculation with a Jinja type calculation: we can hide a family
   - we know how to describe a calcuation
   - we know how to output some documentation (in markdown or in html) of the :term:`configuration`
   - we can calculate with a dynamic family identifer

   **keywords**
   
   - Jinja type calcuation for a hidden property
   - Jinja calcuation's description
   - identifier type parameter
   
   Here we have come to the possibility of making any kind of calculations based on the state of the :term:`configuration`.
   This is an important feature to manage the stateful aspect of a configuration.