rougail/docs/tutorial/jinja.rst

Playing with Jinja 
====================

.. objectives:: Objectives

   In this section we will learn how to create new ways of calculation.

   Up to now, our only way of dynamically (that is, during the runtime) calculating 
   a value is to point on another variable's value. But this is not the only way. 

   We will learn how to insert `Jinja templating language <https://jinja.palletsprojects.com/en/stable/>`_ into our structure files.
   Please note that we do not intend to template our YAML files. 
   That is a completely different activity. 
   We will simply insert YAML code to calculate the value of a variable, a property, or a parameter.

.. prerequisites:: Prerequisites

    - We assume that Rougail's library is :ref:`installed <installation>` on your computer.

    - It is possible to retrieve the current state of the various Rougail files manipulated in this tutorial step
      by checking out the corresponding tag of the `rougail-tutorials` git repository.
      Each tag corresponds to a stage of progress in the tutorial.
      Of course, you can also decide to copy/paste or download the tutorial files contents while following the tutorial steps.

    If you want to follow this tutorial with the help of the corresponding :tutorial:`rougail-tutorials git repository <src/branch/1.1>`, 
    this workshop page corresponds to the tags :tutorial:`v1.1_070 <src/tag/v1.1_070>` to :tutorial:`v1.1_073 <src/tag/v1.1_073>` 
    in the repository.

    ::
   
       git clone https://forge.cloud.silique.fr/stove/rougail-tutorials.git
       git switch --detach v1.1_070


.. glossary::

   calculation

       A calculation is the act of calculating the value of a variable or a property of a variable.
       This calculation can be a redirection of the value of an existing variable (or property), 
       or it can be a custom calculation using the Jinja templating language.

.. type-along:: Why the Jinja templating engine ?
 
We are going to embed some `Jinja templating code <https://jinja.palletsprojects.com/en/stable/>`_ in our structure file.

.. glossary::

    Jinja

        `Jinja <https://jinja.palletsprojects.com>`_ is a template engine.
        we are using Jinja in a classical way, that is, Jinja allows us to handle different cases,
        for example with the `if` statement.

 
.. questions:: What about a `Jinja` calculation?

The :term:`Jinja` templating language enables some complex calculation. 
For example we can code more complex behavior for hiding or disabling a variable, that is 
not only depending on the value of another variable as we saw before.

A conditional hidden family with Jinja
---------------------------------------

Here we are going to use the Jinja conditional (testing) statement.

Here is our structure file:

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/firefox/20-manual.yml
   :language: yaml 
   :caption: The :file:`firefox/20-manual.yml` structure file with some `jinja` code in the `hidden` property

..
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          jinja: |-
            {% if _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: HTTPS
    ...

In this Jinja code, we are testing the `use_for_https` variable:

.. code-block:: jinja

     {% if _.use_for_https %}
         HTTPS is same has HTTP
     {% endif %}

If this variable is set to `true`, the `"HTTPS is same has HTTP"` expression 
will appear.

.. note:: Have a look at the `Jinja website's test section <https://jinja.palletsprojects.com/en/stable/templates/#tests>`_
          for a closer look about testing a variable.

But first, let's set our `use_for_https` variable to `false`: 

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/config.yml
   :language: yaml 
   :caption: The :file:`config/01/config.yml` user data file with the `use_for_https` set to `false`

..
    ---
    proxy_mode: Manual proxy configuration
    manual:
      http_proxy:
        address: http.proxy.net
        port: 3128
      use_for_https: false
      https_proxy:
        address: https.proxy.net
   
Nothing special appears when we launch the Rougail CLI:

.. raw:: html 
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/cmd_ro.txt
   :class: terminal
   
..
    rougail -m firefox/ -u yaml -yf config/01/config.yml

We have this output: 

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/output_ro.html
   :class: output

.. 
    Variables:
    ┣━━ 📓 Configure Proxy Access to the Internet: Manual proxy configuration ◀ loaded from the YAML file "config/01/config.yml" (⏳ No proxy)
    ┗━━ 📂 Manual proxy configuration
        ┣━━ 📂 HTTP Proxy
        ┃   ┣━━ 📓 HTTP address: http.proxy.net ◀ loaded from the YAML file "config/01/config.yml"
        ┃   ┗━━ 📓 HTTP Port: 3128 ◀ loaded from the YAML file "config/01/config.yml" (⏳ 8080)
        ┣━━ 📓 Also use this proxy for HTTPS: false ◀ loaded from the YAML file "config/01/config.yml" (⏳ true)
        ┣━━ 📂 HTTPS Proxy
        ┃   ┣━━ 📓 HTTPS address: https.proxy.net ◀ loaded from the YAML file "config/01/config.yml" (⏳ http.proxy.net)
        ┃   ┗━━ 📓 HTTPS port: 3128
        ┗━━ 📂 SOCKS Proxy
            ┣━━ 📓 SOCKS address: http.proxy.net
            ┣━━ 📓 SOCKS port: 3128
            ┗━━ 📓 SOCKS host version used by proxy: v5

If we set the `use_for_https` to `true` in the :file:`config.yml` user data file, 
then we launch again the Rougail CLI:

.. raw:: html 
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_070/config/01/cmd_ro.txt
   :class: terminal

then we have a warning:

::

    🔔 Warning
    ┗━━ Manual proxy configuration
        ┗━━ HTTPS Proxy
            ┗━━ HTTPS address: 🔔 family "HTTPS Proxy" has property hidden, 
            so cannot access to "HTTPS address", 
            it will be ignored when loading from the YAML file "config/01/config.yml"

What is interesting here is to launch the Rougail CLI in the :term:`read write mode`:

.. code-block:: bash
   :class: terminal 

    rougail -m firefox/ --cli.root manual.https_proxy -u yaml \
            -yf config/01/config.yml --cli.read_write

Then we have an error:

.. code-block:: bash

    ERROR: cannot access to optiondescription "HTTPS Proxy" 
    because has property "hidden" 
    (HTTPS is same has HTTP)

.. note:: Note that we can see the "HTTPS is same has HTTP" string (the one that was present in the Jinja code) 
          as an explanation inside the error message.

.. type-along:: Without calculation

Let's reason on the previous HTTPS proxy configuration's manual mode settings:

.. code-block:: yaml

      use_for_https:
        default: true

      https_proxy:
        type: family
        description: HTTPS Proxy
        hidden:
          variable: _.use_for_https

We see here that the `https_proxy` family is hidden 
depending on the value of another variable. This is what we saw before, 
how to hide a variable.

Now we can code the same behavior in a somehow different way: 

.. code-block:: yaml

      https_proxy':
        description: HTTPS Proxy
        hidden:
          jinja: |-
            {% if _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}            

This code has the same result and yes, it's done in a more complicated way.
We have replaced this simple `hidden` property variable parameter by 
a Jinja parameter. The hidden process is done by a calculation. 

The fact is that it has same result, but it opens more possibilities,
we will see them later.

Jinja with a description
-----------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the `v1.1_071` version::
     
          git switch --detach v1.1_071

It is preferable to include a description related to the Jinja calcuation. 
That allows the calculation performed by the Jinja code to be understood rapidely 
without having to read the code itself. 
This is useful for the :term:`integrators <integrator>` who review these :term:`structure files <structure file>`.

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_071/firefox/20-manual.yml
   :language: yaml 
   :caption: The :file:`firefox/20-manual.yml` structure file with a description added to the `Jinja` code

..
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          jinja: |-
            {% if _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}
          description: in HTTPS case if "_.use_for_https" is set to "true"
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: HTTPS
    ...

This description will appear `in the documentation output for the family <https://forge.cloud.silique.fr/stove/rougail-tutorials/src/tag/v1.1_071#https-proxy-or-socks-proxy>`_

With this rougail command:

.. code-block:: bash

    rougail -m firefox -o doc --doc.output_format html > readme.html

It outputs an HTML report, here is a part of it about the `https_proxy` with the `hidden` property:

.. raw:: html
   :class: output 
       
    manual.<i>https</i>_proxy <br/>

    <b>Hidden</b>: in HTTPS case if "manual.use_for_https" is set to "true" <br/>

    <b>Identifiers</b>: 
    <ul><li>HTTPS</li>

We can see that the hidden explanation for the `manual.https_proxy` family 
is the `hidden` property's description.

Jinja with a parameter
---------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the `v1.1_072` version::
     
          git switch --detach v1.1_072

Regarding dynamic families as in the use case we are dealing with, here is an interesting Rougail feature:
it is possible to retrieve the family identifiers and declare them as parameters in the Jinja calculation.

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_072/firefox/20-manual.yml
   :language: yaml 
   :caption: The :file:`firefox/20-manual.yml` structure file with an identifier type parameter in the hidden property

.. 
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          jinja: |-
            {% if my_identifier == 'HTTPS' and _.use_for_https %}
              HTTPS is same has HTTP
            {% endif %}
          description: in HTTPS case if "_.use_for_https" is set to "true"
          params:
            my_identifier:
              type: identifier
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: HTTPS
    ...

We have added an `identifier` type parameter to the `hidden` property:

.. code-block:: yaml

    hidden:
      params:
        my_identifier:
          type: identifier

Thus we can refine our Jinja code and hide the `https_proxy` dynamic family,
that is only in the case where the identifier is `HTTPS`.

.. code-block:: yaml

    hidden:
      jinja: |-
        {% if my_identifier == 'HTTPS' and _.use_for_https %}
          HTTPS is same has HTTP
        {% endif %}        

In this Jinja code, we can understand that only the `https_proxy` dynamic family 
will be hidden when the `_.use_for_https` variable is set to `true`.

Let's set the `_.use_for_https` variable to `true`.
We have a warning:

.. code-block:: text

    🔔 Warning
    ┗━━ Manual proxy configuration
        ┗━━ HTTPS Proxy
            ┗━━ HTTPS address: 🔔 family "HTTPS Proxy" has property hidden, 
                so cannot access to "HTTPS address", it will be ignored 
                when loading from the YAML file "config/01/config.yml"

It's helpful to run this command in read/write mode: 

.. code-block:: bash
   :class: terminal 

   rougail -m firefox/ --cli.root manual.https_proxy -u yaml \
           -yf config/01/config.yml --cli.read_write

because at that moment we get an error:

.. code-block:: bash
   :class: output

   ERROR: cannot access to optiondescription "HTTPS Proxy" 
   because has property "hidden" (HTTPS is same has HTTP)

So the Jinja code worked well since the `https_proxy` family is hidden only when 
the `use_for_https` variable is set to `true` and the identifier is HTTPS 
(the `socks_proxy` family, however, is not hidden).

Jinja could returns a boolean
---------------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the `v1.1_073` version::
     
          git switch --detach v1.1_073

So far, we have used a Jinja expression that could have a `True` value, 
otherwise it returned nothing (i.e., None, that is a `False` value):

.. code-block:: jinja

    {% if my_identifier == 'HTTPS' and _.use_for_https %}
      HTTPS is same has HTTP
    {% endif %}

We have implicitly used the truthiness capabilities of Jinja, because the 
`HTTPS is same has HTTP` expression is interpreted by Jinja as the `True` value and 
the `""` (empty string) expression is interpreted by Jinja as the `False` value.
This behavior is known as implicit boolean coercion (or truthiness).

.. type-along:: The truthiness of python or Jinja

In Python or in Jinja, **every value has an inherent Boolean interpretation**—it 
can be treated as either `True` or `False` in contexts that expect a Boolean 
(like `if`, `while`, or logical operators). 
This is often referred to as “truthiness.” 
However, **not every value is an instance of the `bool` type**; 
the `bool` type itself has only two instances: `True` and `False`.

.. rubric:: Truthiness Rules

- A value is considered **falsy** if it evaluates to `False` in a Boolean context.  
- Otherwise, it is **truthy**.

.. rubric:: Common Falsy Values

- `None`
- `False`
- Zero of any numeric type: `0`, `0.0`, `0j`, `Decimal(0)`, `Fraction(0, 1)`
- Empty sequences/collections: `''`, `[]`, `()`, `{}`, `set()`, `range(0)`
- Objects that define `__bool__()` returning `False`, or `__len__()` returning `0`

Everything else is truthy, including:

- Non‑zero numbers
- Non‑empty strings, lists, tuples, dictionaries, etc.
- Most user‑defined class instances (unless they override `__bool__` or `__len__`)

.. rubric:: Using truthiness in Code

You can directly use any value in a conditional:

.. code-block:: python

    if some_value:
        print("Truthy")
    else:
        print("Falsy")

.. type-along:: Jinja calcuation without truthiness

This is truthiness capability is entirely possible with Rougail 
but, in programming language theory, 
it is a form of **implicit** type conversion 
where a value of any type is automatically 
interpreted as a boolean in contexts that expect one (e.g., conditionals).

We like things to be declared explicitly. 
It is a better practice to simply avoid this implicit coercion 
and instead:

- return a boolean value as the result of the Jinja expression
- explicitly declare that this expression shall return a boolean type

Here is a Jinja code that return a boolean value:


.. code-block:: jinja

    {{ my_identifier == 'HTTPS' and _.use_for_https }}

And in order to declare explicitly the use of this explicit non-truthiness pratice, 
we add a `return_type` parameter in the `hiden` property:


.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_073/firefox/20-manual.yml
   :language: yaml 
   :caption: The :file:`firefox/20-manual.yml` structure file with an explicit boolean declaration


.. 
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      '{{ identifier }}_proxy':
        description: '{{ identifier }} Proxy'
        hidden:
          jinja: |-
            {{ my_identifier == 'HTTPS' and _.use_for_https }}
          return_type: boolean
          description: in HTTPS case if "_.use_for_https" is set to "true"
          params:
            my_identifier:
              type: identifier
        dynamic:
          - HTTPS
          - SOCKS

        address:
          description: '{{ identifier }} address'
          default:
            variable: __.http_proxy.address

        port:
          description: '{{ identifier }} port'
          default:
            variable: __.http_proxy.port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
          disabled:
            type: identifier
            when: HTTPS
    ...


.. questions:: What about the error message?

    If no text is returned by the Jinja calculation, what about the error message (in case of an error)?

    In case of an error, it is the jinja's description that will appear. 
    
    In our example: `in HTTPS case if "_.use_for_https" is set to "true"`

.. keypoints:: Key points

   We have seen that the `disabled` or `hidden` properties could be calculated.
   The default values can be calculated too. We'll see a bunch of examples later on.

   **progress**
      
   - calculation with a Jinja type calculation: we can hide a family
   - we know how to describe a calcuation
   - we know how to output some documentation (in markdown or in html) of the :term:`configuration`
   - we can calculate with a dynamic family identifer
   - we can set an explicit boolean jinja expression in a jinja calcuation

   **keywords**
   
   - Jinja type calcuation for a hidden property
   - Jinja calcuation's description
   - identifier type parameter
   
   Here we have come to the possibility of making any kind of calculations based on the state of the :term:`configuration`.
   This is an important feature to manage the stateful aspect of a configuration.