Playing with Jinja 
====================

.. objectives:: Objectives

   In this section we will learn how to create new ways of calculation.

   Up to now, our only way of dynamically (that is, during the runtime) calculating 
   a value is to point on another variable's value. But this is not the only way. 

   We will learn how to insert `Jinja templating language <https://jinja.palletsprojects.com/en/stable/>`_ into our structure files.
   Please note that we do not intend to template our YAML files. 
   That is a completely different activity. 
   We will simply insert YAML code to calculate variable, property, or parameter values.


.. prerequisites:: Prerequisites

    - We assume that Rougail's library is :ref:`installed <installation>` on your computer.

    - It is possible to retrieve the current state of the various Rougail files manipulated in this tutorial step
      by checking out the corresponding tag of the `rougail-tutorials` git repository.
      Each tag corresponds to a stage of progress in the tutorial.
      Of course, you can also decide to copy/paste or download the tutorial files contents while following the tutorial steps.

    If you want to follow this tutorial with the help of the corresponding :tutorial:`rougail-tutorials git repository <src/branch/1.1>`, 
    this workshop page corresponds to the tags :tutorial:`v1.1_070 <src/tag/v1.1_070>` to :tutorial:`v1.1_072 <src/tag/v1.1_072>` 
    in the repository.

    ::
   
       git clone https://forge.cloud.silique.fr/stove/rougail-tutorials.git
       git switch --detach v1.1_070

.. type-along:: Why the jinja templating engine ?
 
We are going to embed some `jinja templating code <https://jinja.palletsprojects.com/en/stable/>`_ in our structure file.
 
.. questions:: What about a `Jinja` calculation?

The :term:`Jinja` templating language enables some complex calculation. 
We can code more complex behavior that hiding or disabling a variable depending on the value of another variable,
as we saw before.


.. glossary::

    Jinja

        `Jinja <https://jinja.palletsprojects.com>`_ is a template engine.
        we are using Jinja in a classical way, that is, Jinja allows us to handle different cases,
        for example with the `if` statement.

A conditional hidden family with Jinja
---------------------------------------

Here we are going to use the Jinja conditional (testing) statement.

Here is our structure file:

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/firefox/20-manual.yml
   :language: yaml 
   :caption: The :file:`firefox/20-manual.yml` structure file with some `jinja` code in the `hidden` property

..
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          jinja: |-
            {% if _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: HTTPS
    ...

In this jinja code, we are testing the `use_for_https` variable:

.. code-block:: jinja

     {% if _.use_for_https %}
         HTTPS is same has HTTP
     {% endif %}

If this variable is set to `true`, the `"HTTPS is same has HTTP"` expression 
will appear.

.. note:: Have a look at the `jinja website's test section <https://jinja.palletsprojects.com/en/stable/templates/#tests>`_
          for a closer look about testing a variable.

But first, let's set our `use_for_https` variable to `false`: 

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/config.yml
   :language: yaml 
   :caption: The :file:`config/01/config.yml` user data file with the `use_for_https` set to `false`

..
    ---
    proxy_mode: Manual proxy configuration
    manual:
      http_proxy:
        address: http.proxy.net
        port: 3128
      use_for_https: false
      https_proxy:
        address: https.proxy.net
   
Nothing special appears when we launch the Rougail CLI:

.. raw:: html 
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/cmd_ro.txt
   :class: terminal
   
..
    rougail -m firefox/ -u yaml -yf config/01/config.yml

We have this output: 

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/output_ro.html
   :class: output

.. 
    Variables:
    ┣━━ 📓 Configure Proxy Access to the Internet: Manual proxy configuration ◀ loaded from the YAML file "config/01/config.yml" (⏳ No proxy)
    ┗━━ 📂 Manual proxy configuration
        ┣━━ 📂 HTTP Proxy
        ┃   ┣━━ 📓 HTTP address: http.proxy.net ◀ loaded from the YAML file "config/01/config.yml"
        ┃   ┗━━ 📓 HTTP Port: 3128 ◀ loaded from the YAML file "config/01/config.yml" (⏳ 8080)
        ┣━━ 📓 Also use this proxy for HTTPS: false ◀ loaded from the YAML file "config/01/config.yml" (⏳ true)
        ┣━━ 📂 HTTPS Proxy
        ┃   ┣━━ 📓 HTTPS address: https.proxy.net ◀ loaded from the YAML file "config/01/config.yml" (⏳ http.proxy.net)
        ┃   ┗━━ 📓 HTTPS port: 3128
        ┗━━ 📂 SOCKS Proxy
            ┣━━ 📓 SOCKS address: http.proxy.net
            ┣━━ 📓 SOCKS port: 3128
            ┗━━ 📓 SOCKS host version used by proxy: v5

If we set the `use_for_https` to `true` in the :file:`config.yml` user data file, 
then we launch again the Rougail CLI:

.. raw:: html 
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/cmd_ro.txt
   :class: terminal

then we have a warning:

::

    🔔 Warning
    ┗━━ Manual proxy configuration
        ┗━━ HTTPS Proxy
            ┗━━ HTTPS address: 🔔 family "HTTPS Proxy" has property hidden, 
            so cannot access to "HTTPS address", 
            it will be ignored when loading from the YAML file "config/01/config.yml"

It is interesting here to launch the Rougail CLI in the :term:`read write mode`:

::
   
    rougail -m firefox/ --cli.root manual.https_proxy -u yaml environment \
            -yf config/01/config.yml --cli.read_write

Then we have an error:

.. code-block:: bash

    ERROR: cannot access to optiondescription "HTTPS Proxy" 
    because has property "hidden" 
    (HTTPS is same has HTTP)

FIXME

We can hide or disable some variables or families with other techniques than 
pointing on a variable's value.

Let's reason on the previous HTTPS proxy configuration's manual mode:

.. code-block:: yaml

    manual:

      use_for_https:
        description: Also use this proxy for HTTPS
        default: true

      https_proxy:
        type: family
        description: HTTPS Proxy
        hidden:
          variable: manual.use_for_https


This is extracted from the proxy's manual configuration we discussed before.

We see here that there is an `https_proxy` family that is going to be hidden 
depending on the value of another variable:

.. code-block:: yaml

      https_proxy:
        type: family
        hidden:
          variable: manual.use_for_https

Now we could write it like that: 

.. code-block:: yaml

    manual:

      use_for_https:
        description: Also use this proxy for HTTPS
        default: true

      https_proxy:
        description: HTTPS Proxy
        type: family
        hidden:
          type: jinja
          jinja: |
            {% if rougail.manual.use_for_https %}
            the HTTPS Proxy family is hidden 
            {% endif %}

Yes, it's done in a more complicated (but more powerful) way. 
Let's explain this a little: 

We have replaced this simple hidden property declaration:

.. code-block:: yaml

        hidden:
          variable: manual.use_for_https

by this (more complicated) hidden property declaration:

.. code-block:: yaml

        hidden:
          type: jinja
          jinja: |
            {% if rougail.manual.use_for_https %}
            the HTTPS Proxy family is hidden
            {% endif %}

The fact is that it has same result, but here we have more possibilities. 
The hidden process is done by a calculation. 

Another jinja calculation type sample 
---------------------------------------

We can now hide or disable some variables or families with other techniques than 
pointing on a variable's value.

Let's reason upon the proxy's manual configuration we discussed before.
We have the :file:`dict/02-proxy_manual.yml` structure file:

.. code-block:: yaml
   :caption: the :file:`structfile/02-proxy_manual.yml` file

      ---
      version: '1.1'
      manual:
        description: Manual proxy configuration
        type: family
        disabled:
          type: jinja
          jinja: |
            {% if rougail.proxy.proxy_mode != 'Manual proxy configuration' %}
            the proxy mode is not manual
            {% endif %}

.. questions:: Question

   **question**: OK then. What happens when you select the "Manual proxy configuration"?

Here if the user selects the "Manual proxy configuration" proxy mode, 
the the `manual` family will be disabled. This is what the jinja code says. 
Let's explain it more precisely. 


.. note:: The "the proxy mode is not manual" output is be used in the log outputs
          for example while 

What can be calculated?
---------------------------

We have seen that the `disabled` or `hidden` properties could be calculated.
The default values can be calculated too. 

.. todo:: montrer un exemple de valeur par défaut calculées (type jinja)

.. todo:: montrer aussi ici des exemples de calculs de valeurs variables, ce qui est un des usages principaux de jinja

Using jinja in a dynamic family declaration
-----------------------------------------------

Let's come back to the previous section's FIXME `dynamic family example <conditional_hidden_family>`

In a dynamic family, as seen before, you have the possibility to name your identifier. In the classic declaration, 
the identifier's variable is named "identifier" by default. Sounds logical:

.. code-block:: yaml

      "{{ identifier }}_proxy":
        description: "{{ identifier }} Proxy"
        dynamic:
          - HTTPS
          - SOCKS

Here the identifer's variable takes the value of the `dynamic` family parameter.

.. type-along:: Using a jinja calculation with a parameter

We have the possibility to use a given variable variable inside a jinja calculation:

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/tag/v1.1_050/firefox/20-manual.yml
   :language: yaml
   :caption: firefox/20-proxy.yml

.. 
    manual:

      use_for_https:
        description: Also use this proxy for HTTPS
        default: true

      "{{ identifier }}_proxy":
        description: "{{ identifier }} Proxy"
        dynamic:
          - HTTPS
          - SOCKS
        hidden:
          jinja: |
            {% if my_identifier == 'HTTPS' and manual.use_for_https %}
            HTTPS is same has HTTP
            {% endif %}
          params:
            my_identifier:
              type: identifier
          description: |
            in HTTPS case if "manual.use_for_https" is set to True

        address:
          description: "{{ identifier }} address"
          default:
            variable: manual.http_proxy.address

        port:
          description: "{{ identifier }} port"
          default:
            variable: manual.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: 'HTTPS'

This can be done by defining a `my_identifier` variable.

This jinja variable comes from the `params` parameter of the `hidden` property.

.. code-block:: yaml

          params:
            my_identifier:
              type: identifier

Here the `hidden` property's value is defined by a jinja calculation. 
In this jinja calculation we have a `my_identifier` jinja variable. 

.. code-block:: yaml

        hidden:
          jinja: |
            {% if my_identifier == 'HTTPS' and manual.use_for_https %}
            HTTPS is same has HTTP
            {% endif %}
          params:
            my_identifier:
              type: identifier
          description: |
            in HTTPS case if "manual.use_for_https" is set to True

.. type-along:: The `when` and `when_not` parameter notation

.. todo:: ça devrait être sur une autre page. déplacer cela ailleurs

Handling the SOCKS version
----------------------------

Now we need to handle the SOCKS version as show in the firefox configuration screenshot: 

.. image:: images/firefox_soks_version.png


We'll just add a choice variable with a default value and a disabled property: 

.. code-block:: yaml

    version:
      description: SOCKS host version used by proxy
      choices:
        - v4
        - v5
      default: v5
      disabled:
        type: identifier
        when: 'HTTPS'

There is still something new about this YAML, though. It is the `when` parameter 
of the `disabled` property. You have two possible notations: `when` or `when_not`. 
These two notations are just a short hand of what we could express in jinja as 
this code:

.. code-block:: jinja

    {% if identifier == 'HTTPS' %}
    when the identifer equals 'HTTPS' then the SOCKS version is disabled 
    {% endif %}
   
And the `when_not` parameter is just the logical opposite.

Here is the final version of our YAML dynamic family:

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/tag/v1.1_050/firefox/20-manual.yml
   :language: yaml
   :caption: firefox/20-proxy.yml


.. keypoints:: Key points

   **keywords**
      
   - calculation with a jinja type calculation
   - calculation with a `when` or `when_not` parameter
   - defining a jinja internal variable in a jinja calculation

   **progress**

   Here we have come to the possibility of making any kind of calculations based on the state of the :term:`configuration`.
   This is an important feature to manage the stateful aspect of a configuration.