F #227: First version of Dynamo appliance (#228)
This appliance includes the runtime installation of the Dynamo inference framework. Supported parameters: - ONEAPP_DYNAMO_API_PORT: Port where the Dynamo API will be exposed - ONEAPP_DYNAMO_MODEL_ID: Name of the model in Hugging Face - ONEAPP_DYNAMO_MODEL_TOKEN: HF API token - ONEAPP_DYNAMO_ENGINE_NAME: Name of the dynamo engine to use: mistralrs|sglang|llamacpp|vllm|trtllm|echo_full|echo_core. - ONEAPP_DYNAMO_ENGINE_EXTRA_ARGS_JSON: Engine extra args set in JSON format. - ONEAPP_DYNAMO_ENGINE_EXTRA_ARGS_JSON_BASE64: Engine extra args set in JSON and encoded in base64.
Loading
Please sign in to comment