diff --git a/doc/WebLlm.md b/doc/WebLlm.md index e404bd2..1ecfa6c 100644 --- a/doc/WebLlm.md +++ b/doc/WebLlm.md @@ -20,8 +20,6 @@ No WebGPU → no in-browser LLM can run in this browser. Use a recent Chrome/Edge (≥ 113) or Safari 18+ (chrome://flags/#enable-unsafe-webgpu on Linux). ``` - - --- - [Homepage du machine learning compiler (MLC)](https://llm.mlc.ai/docs/deploy/webllm.html) @@ -30,4 +28,58 @@ Use a recent Chrome/Edge (≥ 113) or Safari 18+ (chrome://flags/#enable-unsafe- --- +- [Homepage Webllm](https://webllm.org/) - [Le projet WebLLM](https://github.com/mlc-ai/web-llm) + + +## install webllm + + +[Suivre l'install](https://github.com/mlc-ai/web-llm) dans le readme. + + +## install the python mlc local chat utility + +### install + +L'install doit se faire avec des version très précises sinon il y a des difficultés. + +- faire un environnement virtuel +- installer de la manière suivante : + +``` +sudo apt-get install git-lfs + +pip install --pre -f https://mlc.ai/wheels \ + mlc-ai-cpu==0.20.0 \ + mlc-llm-cpu==0.20.0.dev0 + + +pip install psutil + +# wget https://github.com/mlc-ai/package/releases/download/v0.9.dev0/mlc_llm_cpu-0.20.0.dev0-py3-none-manylinux_2_28_x86_64.whl +# pip install ./mlc_llm_cpu-0.20.0.dev0-py3-none-manylinux_2_28_x86_64.whl + +pip install apache-tvm-ffi==0.1.11 +# https://pypi.org/project/apache-tvm-ffi/#history +``` + +### usage + +``` +python -m mlc_llm chat HF://mlc-ai/Llama-3-8B-Instruct-q4f16_1-MLC +``` + +``` +You can use the following special commands: + /help print the special commands + /exit quit the cli + /stats print out stats of last request (token/sec) + /metrics print out full engine metrics + /reset restart a fresh chat + /set [overrides] override settings in the generation config. For example, + `/set temperature=0.5;top_p=0.8;seed=23;max_tokens=100;stop=str1,str2` + Note: Separate stop words in the `stop` option with commas (,). + Multi-line input: Use escape+enter to start a new line. +``` +