Unverified Commit 8b186a66 authored by Ruben S. Montero's avatar Ruben S. Montero
Browse files

F #-: New version of Ray appliance 

This version includes the following improvements:

* Support for multi-gpu inference
* Support for OpenAI API to interact with the model
* Support for vLLMs and quantization
* Includes an optional embedded web interface to interact with the deployed LLM
* Updated base appliance to Ubuntu24.04
* Ray and vllm frameworks now run in a Python virtual environment
parent 3a135415
Expand all Show whitespace changes
Inline Side-by-side
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment