gremond/rougail-ai
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

webllm

Browse source
This commit is contained in:
gwen 2026-09-14 21:22:34 +02:00
parent ee66e9fdfa
commit 724bcd8572
8 changed files with 237 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
.gitignore vendored
View file

@ -7,3 +7,4 @@ bribes
site/
.obsidian
.task/
webllm/node_modules/

44
doc/WebLlm.md
View file

@ -34,9 +34,51 @@ Use a recent Chrome/Edge (≥ 113) or Safari 18+ (chrome://flags/#enable-unsafe-
## install webllm
[Suivre l'install](https://github.com/mlc-ai/web-llm) dans le readme.
```
sudo apt install -y nodejs npm
npm install @mlc-ai/web-llm
python -m"http.server" 8000
```
ou bien, lancer le serveur nodejs :
```
npm install -g serve
serve
```
### usage
La méthode principale : le System Prompt. Exemple de script:
```javascript
import { CreateMLCEngine } from "@mlc-ai/web-llm";
// 1. Définir la persona (comme votre instruction SYSTEM dans Ollama)
const systemPrompt = "Tu es un assistant culinaire expert, spécialisé dans la cuisine française. Tu réponds toujours avec enthousiasme et proposes des alternatives aux ingrédients difficiles à trouver. Tu ne parles jamais de sujets hors de la cuisine.";
const messages = [
// 2. Le message système définit le comportement pour toute la conversation
{ role: "system", content: systemPrompt },
{ role: "user", content: "Comment faire une bonne quiche lorraine ?" }
];
// 3. Envoyer la requête
const engine = await CreateMLCEngine("Llama-3.1-8B-Instruct-q4f32_1-MLC");
const reply = await engine.chat.completions.create({ messages });
console.log(reply.choices[0].message.content);
```
- Aller dans le dossier `webllm` et lancer un serveur type `python -m"http.server"`
dans ce dossier.
- Utiliser un browser compatible (j'ai utilisé un-googled-chromium),
le résultat du chat apparaît dans la console.
## install the python mlc local chat utility

14
webllm/index.html Normal file
View file

@ -0,0 +1,14 @@
<!DOCTYPE html>
<html lang="fr">
<head>
<meta charset="UTF-8">
<title>Test WebLLM</title>
</head>
<body>
<h1>Test de WebLLM</h1>
<div id="output"></div>
<!-- Lier votre script JavaScript -->
<script type="module" src="./index.js"></script>
</body>
</html>

55
webllm/index.js Normal file
View file

@ -0,0 +1,55 @@
// import { CreateMLCEngine } from "@mlc-ai/web-llm";
import { CreateMLCEngine } from "./node_modules/@mlc-ai/web-llm/lib/index.js";
// Callback function to update model loading progress
const initProgressCallback = (initProgress) => {
console.log(initProgress);
};
const selectedModel = "Llama-3.1-8B-Instruct-q4f32_1-MLC";
const engine = await CreateMLCEngine(
selectedModel,
{ initProgressCallback: initProgressCallback }, // engineConfig
);
// chat
const messages = [
{ role: "system", content: "You are a helpful AI assistant." },
{ role: "user", content: "Hello!" },
];
// 1. Définir la persona (comme votre instruction SYSTEM dans Ollama)
//const systemPrompt = "Tu es un assistant culinaire expert, spécialisé dans la cuisine française. Tu réponds toujours avec enthousiasme et proposes des alternatives aux ingrédients difficiles à trouver. Tu ne parles jamais de sujets hors de la cuisine.";
//const messages = [
// // 2. Le message système définit le comportement pour toute la conversation
// { role: "system", content: systemPrompt },
// { role: "user", content: "Comment faire une bonne quiche lorraine ?" }
//];
//import { loadPersona } from './personaLoader.js';
// ---
// Chunks is an AsyncGenerator object
const chunks = await engine.chat.completions.create({
messages,
temperature: 0
// stream: true, // <-- Enable streaming
// stream_options: { include_usage: true },
});
let reply = "";
for await (const chunk of chunks) {
reply += chunk.choices[0]?.delta.content || "";
console.log(reply);
if (chunk.usage) {
console.log(chunk.usage); // only last chunk has usage
}
}
const fullReply = await engine.getMessage();
console.log(fullReply);

28
webllm/index.js.1 Normal file
View file

@ -0,0 +1,28 @@
// import { CreateMLCEngine } from "@mlc-ai/web-llm";
import { CreateMLCEngine } from "./node_modules/@mlc-ai/web-llm/lib/index.js";
// Callback function to update model loading progress
const initProgressCallback = (initProgress) => {
console.log(initProgress);
};
const selectedModel = "Llama-3.1-8B-Instruct-q4f32_1-MLC";
const engine = await CreateMLCEngine(
selectedModel,
{ initProgressCallback: initProgressCallback }, // engineConfig
);
// chat
const messages = [
{ role: "system", content: "You are a helpful AI assistant." },
{ role: "user", content: "Hello!" },
];
const reply = await engine.chat.completions.create({
messages,
});
console.log(reply.choices[0].message);
console.log(reply.usage);

5
webllm/package.json Normal file
View file

@ -0,0 +1,5 @@
{
"dependencies": {
"@mlc-ai/web-llm": "^0.2.85"
}
}

75
webllm/persona.txt Normal file
View file

@ -0,0 +1,75 @@
Provided this markdown documentation:
```markdown
| Variable | Description |
|------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **<a id="proxy_mode" name="proxy_mode">proxy_mode</a>**<br/>[`choice`](https://rougail.readthedocs.io/en/latest/variable.html#variables-types) `mandatory` | Configure Proxy Access to the Internet.<br/>**Choices**: <br/>•&nbsp;No proxy **← (default)**<br/>•&nbsp;Auto-detect proxy settings for this network<br/>•&nbsp;Use system proxy settings<br/>•&nbsp;Manual proxy configuration<br/>•&nbsp;Automatic proxy configuration URL |
# Manual proxy configuration
> [!NOTE]
>
> **Path**: manual\
> *`disabled`*\
> **Disabled**: when the variable "[Configure Proxy Access to the Internet](#proxy_mode)" hasn't the value "Manual proxy configuration".
## HTTP Proxy
> [!NOTE]
>
> **Path**: manual.http_proxy
| Variable | Description |
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **<a id="manual.http_proxy.address" name="manual.http_proxy.address">manual.http_proxy.address</a>**<br/>[`domainname`](https://rougail.readthedocs.io/en/latest/variable.html#variables-types) `mandatory` | HTTP proxy address.<br/>**Validators**: <br/>•&nbsp;type domainname<br/>•&nbsp;the domain name can be an IP |
| **<a id="manual.http_proxy.port" name="manual.http_proxy.port">manual.http_proxy.port</a>**<br/>[`port`](https://rougail.readthedocs.io/en/latest/variable.html#variables-types) `mandatory` | HTTP proxy port.<br/>**Validators**: <br/>•&nbsp;well-known ports (1 to 1023) are allowed<br/>•&nbsp;registred ports (1024 to 49151) are allowed<br/>•&nbsp;private ports (greater than 49152) are allowed<br/>**Default**: 8080 |
| Variable | Description |
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------|
| **<a id="manual.use_for_https" name="manual.use_for_https">manual.use_for_https</a>**<br/>[`boolean`](https://rougail.readthedocs.io/en/latest/variable.html#variables-types) `mandatory` | Also use this proxy for HTTPS.<br/>**Default**: true |
## HTTPS Proxy
> [!NOTE]
>
> **Path**: manual.https_proxy\
> *`hidden`*\
> **Hidden**: when the variable "[Also use this proxy for HTTPS](#manual.use_for_https)" has the value "true".
| Variable | Description |
|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **<a id="manual.https_proxy.address" name="manual.https_proxy.address">manual.https_proxy.address</a>**<br/>[`domainname`](https://rougail.readthedocs.io/en/latest/variable.html#variables-types) `mandatory` | HTTPS proxy address.<br/>**Validators**: <br/>•&nbsp;type domainname<br/>•&nbsp;the domain name can be an IP<br/>**Default**: the value of the variable "[HTTP proxy address](#manual.http_proxy.address)". |
| **<a id="manual.https_proxy.port" name="manual.https_proxy.port">manual.https_proxy.port</a>**<br/>[`port`](https://rougail.readthedocs.io/en/latest/variable.html#variables-types) `mandatory` | HTTPS proxy port.<br/>**Validators**: <br/>•&nbsp;well-known ports (1 to 1023) are allowed<br/>•&nbsp;registred ports (1024 to 49151) are allowed<br/>•&nbsp;private ports (greater than 49152) are allowed<br/>**Default**: the value of the variable "[HTTP proxy port](#manual.http_proxy.port)". |
## SOCKS Proxy
> [!NOTE]
>
> **Path**: manual.socks_proxy
| Variable | Description |
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **<a id="manual.socks_proxy.address" name="manual.socks_proxy.address">manual.socks_proxy.address</a>**<br/>[`domainname`](https://rougail.readthedocs.io/en/latest/variable.html#variables-types) | SOCKS proxy address.<br/>**Validators**: <br/>•&nbsp;type domainname<br/>•&nbsp;the domain name can be an IP |
| **<a id="manual.socks_proxy.port" name="manual.socks_proxy.port">manual.socks_proxy.port</a>**<br/>[`port`](https://rougail.readthedocs.io/en/latest/variable.html#variables-types) `mandatory` *`disabled`* | SOCKS proxy port.<br/>**Validators**: <br/>•&nbsp;well-known ports (1 to 1023) are allowed<br/>•&nbsp;registred ports (1024 to 49151) are allowed<br/>•&nbsp;private ports (greater than 49152) are allowed<br/>**Default**: 1080<br/>**Disabled**: when the variable "[SOCKS proxy address](#manual.socks_proxy.address)" has the value "null". |
| **<a id="manual.socks_proxy.version" name="manual.socks_proxy.version">manual.socks_proxy.version</a>**<br/>[`choice`](https://rougail.readthedocs.io/en/latest/variable.html#variables-types) `mandatory` *`disabled`* | SOCKS host version used by proxy.<br/>**Choices**: <br/>•&nbsp;v4<br/>•&nbsp;v5 **← (default)**<br/>**Disabled**: when the variable "[SOCKS proxy address](#manual.socks_proxy.address)" has the value "null". |
| Variable | Description |
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **<a id="auto" name="auto">auto</a>**<br/>[`web address`](https://rougail.readthedocs.io/en/latest/variable.html#variables-types) `mandatory` *`disabled`* | Automatic proxy configuration URL.<br/>**Validators**: <br/>•&nbsp;well-known ports (1 to 1023) are allowed<br/>•&nbsp;registred ports (1024 to 49151) are allowed<br/>•&nbsp;type domainname<br/>•&nbsp;the domain name can be a hostname<br/>**Disabled**: when the variable "[Configure Proxy Access to the Internet](#proxy_mode)" hasn't the value "Automatic proxy configuration URL". |
| **<a id="no_proxy" name="no_proxy">no_proxy</a>**<br/>[`domainname`](https://rougail.readthedocs.io/en/latest/variable.html#variables-types) `multiple` *`disabled`* `unique` | Address for which proxy will be desactivated.<br/>Connections to localhost, 127.0.0.1/8 and ::1 are never proxied.<br/>**Validators**: <br/>•&nbsp;type domainname<br/>•&nbsp;the domain name can starts by a dot<br/>•&nbsp;the domain name can be a hostname<br/>•&nbsp;the domain name can be an IP<br/>•&nbsp;the domain name can be network in CIDR format<br/>**Examples**: <br/>•&nbsp;.mozilla.org<br/>•&nbsp;.net.nz<br/>•&nbsp;192.168.1.0/24<br/>**Disabled**: when the variable "[Configure Proxy Access to the Internet](#proxy_mode)" has the value "No proxy". |
```
This markdown describes an organised data set.
It describes some kind of a storage content with variables names.
These variables are organised into groups of data in a tree-like structure, that is Some variables can contain variables.
Like a folder structure in a hard disk, you have long names and short names.
When the user talks about a variable's name, he means the long name.
The user want to retrieve a variable name (or a list of variable names) that have a given role, along with an understanding of the variable itself.
You help the user in:
- retieving the variables
- understanding the retrieved variable's role.
That's like some kind of a database storage that returns all or part of its data structure depending on the user's request.
You can retrive the variables corresponding to the user's query because we're dealing here with a consistent data structure.
If the user asks you for information about a particular variable, provide him with that information.
In summary, you act as a query tool for a data structure that is described in the markdown documentation provided to you.

16
webllm/personaLoader.js Normal file
View file

@ -0,0 +1,16 @@
// personaLoader.js
export async function loadPersona(filepath = './persona.txt') {
try {
const response = await fetch(filepath);
if (!response.ok) {
throw new Error(`Erreur ${response.status}: impossible de charger la persona`);
}
const systemPrompt = await response.text();
return systemPrompt.trim();
} catch (error) {
console.error("Erreur chargement persona:", error.message);
throw error;
}
}