.. _tutorial_custom_type:

Custom type
============

.. objectives:: Objectives

    In this sections we are going to learn how to create custom types with Rougail.
    In short, a custom type is nothing more than a kind of a template for a variable or a family.

.. prerequisites:: Prerequisites

    - We assume that Rougail's library is :ref:`installed <installation>` on your computer.

    - It is possible to retrieve the current state of the various Rougail files manipulated in this tutorial step
      by checking out the corresponding tag of the `rougail-tutorials` git repository.
      Each tag corresponds to a stage of progress in the tutorial.
      Of course, you can also decide to copy/paste or download the tutorial files contents while following the tutorial steps.

    If you want to follow this tutorial with the help of the corresponding :tutorial:`rougail-tutorials git repository <src/branch/1.1>`,
    this workshop page corresponds to the tags :tutorial:`v1.1_080 <src/tag/v1.1_080/README.md>` to :tutorial:`v1.1_085 <src/tag/v1.1_085/README.md>`
    in the repository.

    ::

       git clone https://forge.cloud.silique.fr/stove/rougail-tutorials.git
       git switch --detach v1.1_080

HTTP Proxy with "proxy" type
--------------------------------

Let's look now at the possibilities for adding types.
This feature is essential for having a truly adaptable tool.

.. note:: It is actually possible to add types,
          but that involves adding predefined types to the :term:`Tiramisu` underlying consistency library itself,
          and that’s a different matter altogether. There is a simpler way to achieve this with the Rougail type definition feature.

It is possible in Rougail to create custom types in a very simple way, it is much like defining variables or families.

Our folder structure will be expanded a bit:

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_080/tree.html

Notice that we have now a new :file:`types` folder in our tree structure.
This is where we will place our new `proxy` type definition.

.. note:: Choosing the :file:`types/proxy/00-type.yml` file name is not mandatory.
          You can name this folder whatever you want.
          It is simply a good practice to separate structure files from type definition files.
          We will see later on how to indicate to Rougail that a file contains a type declaration.

In accordance with our :ref:`naming policy <namingconvention>`, we have created a file named :file:`00-type.yml`
inside the `proxy` folder.
Let's examine this more closely this :file:`types/proxy/00-type.yml` type definition file:

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_080/types/proxy/00-type.yml
   :language: yaml
   :caption: The :file:`types/proxy/00-type.yml` type structure file that defines our new `proxy` type.

..
    %YAML 1.2
    ---
    version: 1.1

    proxy:

      address:
        description: Proxy address
        type: domainname
        params:
          allow_ip: true

      port:
        description: Proxy port
        type: port
        default: 8080
    ...

The new type named `proxy` is declared the same way we declare a family.
It's more or less like a family declaration except it will be used as a type definition.
If you remember, it's mostly the content as the family `manual.http_proxy` in `firefox/10-manual.yml` file of this tutorial.

.. questions:: How do we use a new type?

    Very simply. In the usual way: we will add a `type` parameter in our family declaration.

Let's modify the existing family using this new `proxy` type:

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_080/firefox/10-manual.yml
   :language: yaml
   :caption: The :file:`firefox/10-manual.yml` structure file with less code due to the `proxy` type definition

..
    %YAML 1.2
    ---
    version: 1.1

    manual:
      description: Manual proxy configuration
      disabled:
        variable: _.proxy_mode
        when_not: Manual proxy configuration

      http_proxy:
        description: HTTP Proxy
        type: proxy
    ...

We can see that the type declared for the `http_proxy` family is now `type: proxy`.
It's perfectly natural in the Rougail style. 

Due to the type definition we can "omit" some settings in the structure file
such as the `domainname` type and the `allow_ip` parameter. 
And the `address` variable it is not necessary to be specified at all in the structure file
because it's in the type definition.

In order to use our new type we need to declare it to Rougail otherwise the new type is not available.
The Rougail CLI has the `--types` type declaration command line option for this purpose.
Let's launch the Rougail CLI with this command line parameter:

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_080/config/01/cmd_ro.txt
   :class: terminal

The CLI output is entirely standard and does not specifically mention the call to a new type.
The result is the same as usual, as if the type declaration were omitted, which is what we want.
We want the same behavior as usual.

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_080/config/01/output_ro.html
   :class: output

HTTPS and SOCKS Proxy with "proxy" type
---------------------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the :tutorial:`v1.1_081 <src/tag/v1.1_081/README.md>` version:

          git switch --detach v1.1_081

With this use of a type, it is possible to define our HTTPS and SOCKS Proxy in a much more conscious way:
It's very practical. In fact, the type definition adds nothing else to the structural logic.

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_081/firefox/20-manual.yml
   :language: yaml
   :caption: The :file:`firefox/20-manual.yml` structure file with less code due to the `proxy` type definition

..
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      https_proxy:
        description: HTTPS Proxy
        type: proxy
        hidden:
          variable: _.use_for_https

      socks_proxy:
        description: SOCKS Proxy
        type: proxy
    ...

Now we have two families with the `proxy` type: `https_proxy` and `socks_proxy`.

But we haven't recovered everything we had before.
Using a type definition is fine, but we still need to find exactly the configuration situation 
we had before using the type definition.

Add a variable in a family with custom type
--------------------------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the :tutorial:`v1.1_082 <src/tag/v1.1_082/README.md>` version::

          git switch --detach v1.1_082


In the `socks_proxy` family definition, we need to added a new variable, the `version` variable:

.. code-block:: yaml

    version:
      description: SOCKS host version used by proxy
      choices:
        - v4
        - v5
      default: v5

You can see that what we call a type definition in Rougail is very flexible, because it is perfectly possible
to add to what has been defined in the type.
The type definition here is therefore not only a template but much more an extensible schema.

Redefine default value in custom type variable
-----------------------------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the :tutorial:`v1.1_083 <src/tag/v1.1_083/README.md>` version::

          git switch --detach v1.1_083

In this section we are going to make an implicit :ref:`redefinition <redefine>` of variables default values.

.. note::

   We've always said that the redefine is explicit. But there's an exception here.
   There are two types of redefine for a custom type, an implicit and an explicit.
   An implicit redefinition is a redefinition which has not been declared with the `redefine` attribute.
   An explicit redefinition is a redefinition which has been declared with the `redefine: true` attribute.

   Only default values can be implicitly redefined (and only in the type definitions), all other attributes
   need to be explicitely redefined.

In the type definition file, we are now setting default values for the `address` and `port` variables:

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_083/types/proxy/00-type.yml
   :language: yaml
   :caption: The :file:`types/proxy/00-type.yml` type definition file with the default values for `address` and `port`

..
    %YAML 1.2
    ---
    version: 1.1

    proxy:

      address:
        description: Proxy address
        type: domainname
        params:
          allow_ip: true
        default:
          variable: __.http_proxy.address

      port:
        description: Proxy port
        type: port
        default:
          variable: __.http_proxy.port
    ...

But we don't want theses default values for HTTP definition. We can :term:`redefine` these type settings in the structure file:

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_083/firefox/10-manual.yml
   :language: yaml
   :caption: The :file:`firefox/10-manual.yml` structure definition file with the **redefined** default values for `address` and `port`

..
    %YAML 1.2
    ---
    version: 1.1

    manual:
      description: Manual proxy configuration
      disabled:
        variable: _.proxy_mode
        when_not: Manual proxy configuration

      http_proxy:
        description: HTTP Proxy
        type: proxy

        address:
          default: null

        port:
          default: 8080
    ...

This means that the default values for these variables will now be those defined
in the structure file if the values are defined (and not in the type definitions file).

Let's have a closer look at the behavior here.

Let's launch the Rougail CLI:

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_082/config/01/cmd_ro.txt
   :class: terminal

..
    rougail -m firefox/ -u yaml -yf config/01/config.yml

We have this output:

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_082/config/01/output_ro.html
   :class: output

We can see in the output that

- the default values of the `port` variable, which is `8080` in the structure file, comes from the strucure file.
- the default values of the `address` variable is not set because it is `null` by default in the structure file,
  so there is not such thing as a default value set for this variable.

.. questions:: What happens if the default value in the type definition is not suitable for us?

If we hadn't defined a value in the userdata file, the Rougail CLI would have returned the following error:

::

    🛑 Caution
    ┗━━ Manual proxy configuration
        ┣━━ HTTP Proxy
        ┃   ┗━━ Proxy address: 🛑 mandatory variable but has no value
        ┗━━ SOCKS Proxy
            ┗━━ Proxy address: 🛑 mandatory variable but has no value

Therefore, the default value that was taken into account is indeed that of the structure file, and not that of the type definition.

.. note:: We can see that the Rougail behavior is always very simple, there is no such thing as an MRO
          (Method Resolution Object) that try fallback to the default value defined in the type definition file.
          The Rougail behavior is much more simple and transparent.

.. type-along:: kinematics of setting the `address` value

Let's take the time to explain how the value of the variable `manual.http_proxy.address` is determined using the type definition.

The :file:`10-manual.yml` structure file defines the :file:`manual.http_proxy` family
which is defined with the help of the `proxy` type:

.. code-block:: yaml

  http_proxy:
    description: HTTP Proxy
    type: proxy

In the :file:`types/proxy/00-type.yml` file we have the `proxy` type definition and its default value setting:

.. code-block:: yaml

    default:
      variable: __.http_proxy.address

This default value setting point to the `http_proxy.address` value in the :file:`firefox/10-manual.yml` structure file:

.. code-block:: yaml

    address:
      default: null

Which is `null` by default but in the :file:`config/01/config.yml` user data file we have the `http.proxy.net` value set:

.. code-block:: yaml

    manual:
      http_proxy:
        address: http.proxy.net

In the CLI standard output we can see that the `http_proxy.address` variable value is
finally set by the user data file:

::

    ┗━━ 📂 manual (Manual proxy configuration)
        ┣━━ 📂 http_proxy (HTTP Proxy)
        ┃   ┣━━ 📓 address (Proxy address): http.proxy.net ◀ loaded from the YAML
        ┃   ┃   file "config/01/config.yml"


Redefine other parameter in custom type for HTTP
--------------------------------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the :tutorial:`v1.1_084 <src/tag/v1.1_084/README.md>` version::

          git switch --detach v1.1_084

In this sections we are going to make explicit :term:`redefinitions <redefine>`.

Note here that, in order to allow for the genericity of a type declaration, 
we have omitted to specify the variable descriptions in the type declaration.
We will therefore need to enter these descriptions in the structure file.

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_084/types/proxy/00-type.yml
   :language: yaml
   :caption: The :file:`types/proxy/00-type.yml` type definition file with the default definitions and no description

..
    %YAML 1.2
    ---
    version: 1.1

    proxy:

      address:
        type: domainname
        params:
          allow_ip: true
        default:
          variable: __.http_proxy.address

      port:
        type: port
        default:
          variable: __.http_proxy.port
    ...

So, as the `address` and `port` variables don't have any `description` attribute,
we need to add one. It is something like an extension of the type definitions here 
in the structure file:

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_084/firefox/10-manual.yml
   :language: yaml
   :caption: The :file:`firefox/10-manual.yml` structure definition file with explicit redefinitions

..
    %YAML 1.2
    ---
    version: 1.1

    manual:
      description: Manual proxy configuration
      disabled:
        variable: _.proxy_mode
        when_not: Manual proxy configuration

      http_proxy:
        description: HTTP Proxy
        type: proxy

        address:
          redefine: true
          default: null
          description: HTTP proxy address

        port:
          redefine: true
          default: 8080
          description: HTTP proxy port
    ...

.. note:: Transforming an implicit defintion into an explicit redefinition is very simple; you just need to add the `redefine: true` attribute:

Let's launch the Rougail CLI:

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_083/config/01/cmd_ro.txt
   :class: terminal

..
    rougail -m firefox/ -u yaml -yf config/01/config.yml

We have this output:

.. raw:: html
   :url: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_083/config/01/output_ro.html
   :class: output

..
    Variables:
    ┣━━ 📓 Configure Proxy Access to the Internet: Manual proxy configuration ◀
    ┃   loaded from the YAML file "config/01/config.yml" (⏳ No proxy)
    ┗━━ 📂 Manual proxy configuration
        ┣━━ 📂 HTTP Proxy
        ┃   ┣━━ 📓 HTTP proxy address: http.proxy.net ◀ loaded from the YAML file
        ┃   ┃   "config/01/config.yml"
        ┃   ┗━━ 📓 HTTP proxy port: 3128 ◀ loaded from the YAML file
        ┃       "config/01/config.yml" (⏳ 8080)
        ┣━━ 📓 Also use this proxy for HTTPS: false ◀ loaded from the YAML file
        ┃   "config/01/config.yml" (⏳ true)
        ┣━━ 📂 HTTPS Proxy
        ┃   ┣━━ 📓 HTTPS proxy address: https.proxy.net ◀ loaded from the YAML file
        ┃   ┃   "config/01/config.yml" (⏳ http.proxy.net)
        ┃   ┗━━ 📓 HTTPS proxy port: 3128
        ┗━━ 📂 SOCKS Proxy
            ┣━━ 📓 SOCKS proxy address: http.proxy.net
            ┣━━ 📓 SOCKS proxy port: 3128
            ┗━━ 📓 SOCKS host version used by proxy: v5

We can see in the output that the `address` and `port` variables have default values,
and descriptions ("HTTP proxy address", "HTTP proxy port") that weren't present in the
type definition file.

Redefine other parameter in custom type for HTTPS and SOCKS
-----------------------------------------------------------

.. type-along:: For those who follow the tutorial with the help of the git repository

    Now you need to checkout the :tutorial:`v1.1_085 <src/tag/v1.1_085/README.md>` version::

          git switch --detach v1.1_085

We're going to do exactly the same modification with the `socks_proxy` family,
that is we will base the `socks_proxy` family on the type definition.

Following the same reasoning, we add a in the structure file some `description` attributes to the `socks_proxy` family 
and also add some explicit redefinition attribute (`redefine: true`).

.. extinclude:: https://forge.cloud.silique.fr/stove/rougail-tutorials/raw/commit/v1.1_085/firefox/20-manual.yml
   :language: yaml
   :caption: The :file:`firefox/20-manual.yml` structure file with the `redefine` and the `description` attribute

..
    %YAML 1.2
    ---
    version: 1.1

    manual:

      use_for_https: true  # Also use this proxy for HTTPS

      https_proxy:
        description: HTTPS Proxy
        type: proxy
        hidden:
          variable: _.use_for_https

        address:
          redefine: true
          description: HTTPS proxy address

        port:
          redefine: true
          description: HTTPS proxy port

      socks_proxy:
        description: SOCKS Proxy
        type: proxy

        address:
          redefine: true
          description: SOCKS proxy address

        port:
          redefine: true
          description: SOCKS proxy port

        version:
          description: SOCKS host version used by proxy
          choices:
            - v4
            - v5
          default: v5
    ...


.. keypoints:: Key points

   - the very practical aspects of type declaration
   - type usage in the Rougail CLI
   - more conscious family definition
   - using a type for an extended family definition
   - we can redefine things in the structure file and it overloads the type definition file
   - implicit or explicit redefinitions

   We have seen how the definition of type, used effectively, allows for a new expressiveness.

