Playing with Jinja 
====================

.. objectives:: Objectives

   In this section we will learn how to create new ways of calculation.

   Up to now, our only way of dynamically (that is, during the runtime) calculating 
   a value is to point on another variable's value. But this is not the only way. 

   We will learn how to insert `Jinja templating language <https://jinja.palletsprojects.com/en/stable/>`_ into our structure files.
   Please note that we do not intend to template our YAML files. 
   That is a completely different activity. 
   We will simply insert YAML code to calculate variable, property, or parameter values.


.. prerequisites:: Prerequisites

    - We assume that Rougail's library is :ref:`installed <installation>` on your computer.

    - It is possible to retrieve the current state of the various Rougail files manipulated in this tutorial step
      by checking out the corresponding tag of the `rougail-tutorials` git repository.
      Each tag corresponds to a stage of progress in the tutorial.
      Of course, you can also decide to copy/paste or download the tutorial files contents while following the tutorial steps.

    If you want to follow this tutorial with the help of the corresponding :tutorial:`rougail-tutorials git repository <src/branch/1.1>`, 
    this workshop page corresponds to the tags :tutorial:`v1.1_070 <src/tag/v1.1_070>` to :tutorial:`v1.1_072 <src/tag/v1.1_072>` 
    in the repository.

    ::
   
       git clone https://forge.cloud.silique.fr/stove/rougail-tutorials.git
       git switch --detach v1.1_070

.. type-along:: Why the jinja templating engine ?
 
We are going to embed some `jinja templating code <https://jinja.palletsprojects.com/en/stable/>`_ in our structure file.

.. glossary::

    Jinja

        `Jinja <https://jinja.palletsprojects.com>`_ is a template engine.
        we are using Jinja in a classical way, that is, Jinja allows us to handle different cases,
        for example with the `if` statement.

 
.. questions:: What about a `Jinja` calculation?

The :term:`Jinja` templating language enables some complex calculation. 
For example we can code more complex behavior for hiding or disabling a variable, that is 
not only depending on the value of another variable as we saw before.

A conditional hidden family with Jinja
---------------------------------------

Here we are going to use the Jinja conditional (testing) statement.

Here is our structure file:

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/firefox/20-manual.yml
   :language: yaml 
   :caption: The :file:`firefox/20-manual.yml` structure file with some `jinja` code in the `hidden` property

..
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          jinja: |-
            {% if _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: HTTPS
    ...

In this jinja code, we are testing the `use_for_https` variable:

.. code-block:: jinja

     {% if _.use_for_https %}
         HTTPS is same has HTTP
     {% endif %}

If this variable is set to `true`, the `"HTTPS is same has HTTP"` expression 
will appear.

.. note:: Have a look at the `jinja website's test section <https://jinja.palletsprojects.com/en/stable/templates/#tests>`_
          for a closer look about testing a variable.

But first, let's set our `use_for_https` variable to `false`: 

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/config.yml
   :language: yaml 
   :caption: The :file:`config/01/config.yml` user data file with the `use_for_https` set to `false`

..
    ---
    proxy_mode: Manual proxy configuration
    manual:
      http_proxy:
        address: http.proxy.net
        port: 3128
      use_for_https: false
      https_proxy:
        address: https.proxy.net
   
Nothing special appears when we launch the Rougail CLI:

.. raw:: html 
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/cmd_ro.txt
   :class: terminal
   
..
    rougail -m firefox/ -u yaml -yf config/01/config.yml

We have this output: 

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/output_ro.html
   :class: output

.. 
    Variables:
    ┣━━ 📓 Configure Proxy Access to the Internet: Manual proxy configuration ◀ loaded from the YAML file "config/01/config.yml" (⏳ No proxy)
    ┗━━ 📂 Manual proxy configuration
        ┣━━ 📂 HTTP Proxy
        ┃   ┣━━ 📓 HTTP address: http.proxy.net ◀ loaded from the YAML file "config/01/config.yml"
        ┃   ┗━━ 📓 HTTP Port: 3128 ◀ loaded from the YAML file "config/01/config.yml" (⏳ 8080)
        ┣━━ 📓 Also use this proxy for HTTPS: false ◀ loaded from the YAML file "config/01/config.yml" (⏳ true)
        ┣━━ 📂 HTTPS Proxy
        ┃   ┣━━ 📓 HTTPS address: https.proxy.net ◀ loaded from the YAML file "config/01/config.yml" (⏳ http.proxy.net)
        ┃   ┗━━ 📓 HTTPS port: 3128
        ┗━━ 📂 SOCKS Proxy
            ┣━━ 📓 SOCKS address: http.proxy.net
            ┣━━ 📓 SOCKS port: 3128
            ┗━━ 📓 SOCKS host version used by proxy: v5

If we set the `use_for_https` to `true` in the :file:`config.yml` user data file, 
then we launch again the Rougail CLI:

.. raw:: html 
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/cmd_ro.txt
   :class: terminal

then we have a warning:

::

    🔔 Warning
    ┗━━ Manual proxy configuration
        ┗━━ HTTPS Proxy
            ┗━━ HTTPS address: 🔔 family "HTTPS Proxy" has property hidden, 
            so cannot access to "HTTPS address", 
            it will be ignored when loading from the YAML file "config/01/config.yml"

What is interesting here is to launch the Rougail CLI in the :term:`read write mode`:

::
   
    rougail -m firefox/ --cli.root manual.https_proxy -u yaml environment \
            -yf config/01/config.yml --cli.read_write

Then we have an error:

.. code-block:: bash

    ERROR: cannot access to optiondescription "HTTPS Proxy" 
    because has property "hidden" 
    (HTTPS is same has HTTP)

.. note:: Note that we can see the "HTTPS is same has HTTP" string (the one that was present in the jinja code) 
          as an explanation inside the error message.


.. type-along:: What can be calculated?

We have seen that the `disabled` or `hidden` properties could be calculated.
The default values can be calculated too. We'll see a bunch of examples later on.

Let's reason on the previous HTTPS proxy configuration's manual mode settings:

.. code-block:: yaml

      use_for_https:
        default: true

      https_proxy:
        type: family
        description: HTTPS Proxy
        hidden:
          variable: _.use_for_https

We see here that the `https_proxy` family is be hidden 
depending on the value of another variable. This is what we saw before, 
how to hide a variable.

Now we can code the same behavior in a somehow different way: 

.. code-block:: yaml

      https_proxy':
        description: HTTPS Proxy
        hidden:
          jinja: |-
            {% if _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}

This code has the same result and yes, it's done in a more complicated way.
We have replaced this simple `hidden` property variable parameter by 
a jinja parameter. The hidden process is done by a calculation. 

The fact is that it has same result, but it opens more possibilities,
we will see them later.

Jinja with a description
-----------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the `v1.1_071` version::
     
          git switch --detach v1.1_071

It is preferable to include a description related to the jinja calcuation. 
That allows the calculation performed by the Jinja code to be understood rapidely 
without having to read the code itself. 
This is useful for the :term:`integrators <integrator>` who review these :term:`structure files <structure file>`.

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_071/firefox/20-manual.yml
   :language: yaml 
   :caption: The :file:`firefox/20-manual.yml` structure file with a description added to the `jinja` code

..
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          jinja: |-
            {% if _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}
          description: in HTTPS case if "_.use_for_https" is set to "true"
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: HTTPS
    ...

This description will appear `in the documentation output for the family <https://forge.cloud.silique.fr/stove/rougail-tutorials/src/tag/v1.1_071#https-proxy-or-socks-proxy>`_

With this rougail command:

.. code-block:: bash

    rougail -m firefox -o doc --doc.output_format html > readme.html

It outputs an HTML report, here is a part of it:

.. code-block:: html

    <i>HTTPS</i> Proxy or <i>SOCKS</i> Proxy

    This family builds families dynamically.

    <b>Path</b>: <ul><li>manual.<i>https</i>_proxy</li>
    <li>manual.<i>socks</i>_proxy</li></ul>

    <mark><i>hidden</i></mark>

    <b>Hidden</b>: in HTTPS case if "manual.use_for_https" is set to "true" <br/>

    <b>Identifiers</b>: 
    <ul><li>HTTPS</li>
    <li>SOCKS</li></ul>

As for the `https_proxy` with the `hidden` property, we have this:

.. raw:: html
   :class: output 
       
    manual.<i>https</i>_proxy <br/>

    <b>Hidden</b>: in HTTPS case if "manual.use_for_https" is set to "true" <br/>

    <b>Identifiers</b>: 
    <ul><li>HTTPS</li>

We can see that the hidden explanation for the `manual.https_proxy` family 
is the `hidden` property's description.




Jinja with a parameter
---------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the `v1.1_072` version::
     
          git switch --detach v1.1_072

Regarding dynamic families as in the use case we are dealing with, here is an interesting Rougail feature:
it is possible to retrieve the family identifiers and declare them as parameters in the Jinja calculation.

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_072/firefox/20-manual.yml
   :language: yaml 
   :caption: The :file:`firefox/20-manual.yml` structure file with an identifier type parameter in the hidden property

.. 
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          jinja: |-
            {% if my_identifier == 'HTTPS' and _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}
          description: in HTTPS case if "_.use_for_https" is set to "true"
          params:
            my_identifier:
              type: identifier
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: HTTPS
    ...

We have added an `identifier` type parameter to the `hidden` property:

.. code-block:: yaml

    hidden:
      params:
        my_identifier:
          type: identifier

Thus we can refine our Jinja code and hide the `https_proxy` dynamic family,
that is only in the case where the identifier is `HTTPS`.

.. code-block:: yaml

    hidden:
      jinja: |-
        {% if my_identifier == 'HTTPS' and _.use_for_https %}
          HTTPS is same has HTTP
        {% endif %}

In this Jinja code, we can understand that only the `https_proxy` dynamic family 
will be hidden when the `_.use_for_https` variable is set to `true`.

FIXME

::

     rougail -m firefox/ -u yaml -yf config/01/config.yml
    ╭────────────── Caption ───────────────╮
    │ Variable Default value               │
    │          Modified value              │
    │          (⏳ Original default value) │
    ╰──────────────────────────────────────╯
    Variables:
    ┣━━ 📓 Configure Proxy Access to the Internet: Manual proxy configuration ◀ loaded from the YAML 
    ┃   file "config/01/config.yml" (⏳ No proxy)
    ┗━━ 📂 Manual proxy configuration
        ┣━━ 📂 HTTP Proxy
        ┃   ┣━━ 📓 HTTP address: http.proxy.net ◀ loaded from the YAML file "config/01/config.yml"
        ┃   ┗━━ 📓 HTTP Port: 3128 ◀ loaded from the YAML file "config/01/config.yml" (⏳ 8080)
        ┣━━ 📓 Also use this proxy for HTTPS: false ◀ loaded from the YAML file "config/01/config.yml" 
        ┃   (⏳ true)
        ┣━━ 📂 HTTPS Proxy
        ┃   ┣━━ 📓 HTTPS address: https.proxy.net ◀ loaded from the YAML file "config/01/config.yml" (⏳
        ┃   ┃   http.proxy.net)
        ┃   ┗━━ 📓 HTTPS port: 3128
        ┗━━ 📂 SOCKS Proxy
            ┣━━ 📓 SOCKS address: http.proxy.net
            ┣━━ 📓 SOCKS port: 3128
            ┗━━ 📓 SOCKS host version used by proxy: v5

si on met à true, on a un warning :

::

    🔔 Warning
    ┗━━ Manual proxy configuration
        ┗━━ HTTPS Proxy
            ┗━━ HTTPS address: 🔔 family "HTTPS Proxy" has property hidden, so cannot access to "HTTPS 
                address", it will be ignored when loading from the YAML file "config/01/config.yml"

    ╭───────────────────── Caption ─────────────────────╮
    │ Variable              Default value               │
    │ Unmodifiable variable Modified value              │
    │                       (⏳ Original default value) │
    ╰───────────────────────────────────────────────────╯
    Variables:
    ┣━━ 📓 Configure Proxy Access to the Internet: Manual proxy configuration ◀ loaded from the YAML 
    ┃   file "config/01/config.yml" (⏳ No proxy)
    ┗━━ 📂 Manual proxy configuration
        ┣━━ 📂 HTTP Proxy
        ┃   ┣━━ 📓 HTTP address: http.proxy.net ◀ loaded from the YAML file "config/01/config.yml"
        ┃   ┗━━ 📓 HTTP Port: 3128 ◀ loaded from the YAML file "config/01/config.yml" (⏳ 8080)
        ┣━━ 📓 Also use this proxy for HTTPS: true ◀ loaded from the YAML file "config/01/config.yml" 
        ┃   (⏳ true)
        ┣━━ 📂 HTTPS Proxy
        ┃   ┣━━ 📓 HTTPS address: http.proxy.net
        ┃   ┗━━ 📓 HTTPS port: 3128
        ┗━━ 📂 SOCKS Proxy
            ┣━━ 📓 SOCKS address: http.proxy.net
            ┣━━ 📓 SOCKS port: 3128
            ┗━━ 📓 SOCKS host version used by proxy: v5



.. keypoints:: Key points

   **keywords**
      
   - calculation with a jinja type calculation
   - calculation with a `when` or `when_not` parameter
   - defining a jinja internal variable in a jinja calculation

   **progress**

   Here we have come to the possibility of making any kind of calculations based on the state of the :term:`configuration`.
   This is an important feature to manage the stateful aspect of a configuration.