CLI Arguments¶
inference¶
Make an inference to ModelZ deployment
inference [OPTIONS] [UNKNOWN]...
Options
- -k, --key <key>¶
Required API key for Modelz, will read from env $MODELZ_API_KEY if not provided
- -e, --endpoint <endpoint>¶
Required Inference endpoint for Modelz deployment
- --serde <serde>¶
Serialization/deserialization method
- Options:
json | msgpack | raw
- --read-stdin¶
Read bytes from stdin
- --write-file <write_file>¶
Write received data to file
Arguments
- UNKNOWN¶
Optional argument(s)
Environment variables
- MODELZ_API_KEY
Provide a default for
--key
metrics¶
Get metric from ModelZ deployment
metrics [OPTIONS]
Options
- -k, --key <key>¶
Required API key for Modelz, will read from env $MODELZ_API_KEY if not provided
- -d, --deployment <deployment>¶
Required Deployment id
Environment variables
- MODELZ_API_KEY
Provide a default for
--key
build¶
Build image by ModelZ builder
build [OPTIONS]
deployment¶
Operate to ModelZ deployments
deployment [OPTIONS] COMMAND [ARGS]...
create¶
deployment create [OPTIONS]
Options
- --host <host>¶
Control Apiserver host for Modelz, will read from env $MODELZ_CTRL_HOST if not provided
- -u, --user-id <user_id>¶
Required login name for Modelz, will read from env $MODELZ_USER if not provided
- -k, --key <key>¶
Required API key for Modelz, will read from env $MODELZ_API_KEY if not provided
- --image-source <image_source>¶
Image pull source
- Options:
docker | huggingface
- --image <image>¶
Required URL of Docker image path or HuggingFace project Path
- --server-resource <server_resource>¶
Required Server Resource used for deployment
- Options:
cpu-4c-16g | nvidia-ada-l4-2-24c-96g | nvidia-ada-l4-4-48c-192g | nvidia-ada-l4-8c-32g | nvidia-ampere-a100-40g-12c-85g | nvidia-tesla-t4-4c-16g
- --framework <framework>¶
Required Framework of deployment
- Options:
gradio | mosec | other | streamlit | unknown
- --name <name>¶
Required Name of deployment
- --min-replicas <min_replicas>¶
MinReplicas is the minimum number of replicas of the deployment
- --max-replicas <max_replicas>¶
MaxReplicas is the maximum number of replicas of the deployment
- --target-load <target_load>¶
TargetLoad is the target load of the deployment. (inflight requests per replica)
- --startup-duration <startup_duration>¶
StartupDuration is the startup timeout
- --zero-duration <zero_duration>¶
ZeroDuration is the idle timeout before scaling to zero
- --http-probe-path <http_probe_path>¶
HTTPProbePath is the user defined path of the http probe
- --port <port>¶
Port is the port of the deployment
- --command <command>¶
Command is the command to run
- --env-vars <env_vars>¶
EnvVars is the environment variables of the deployment, input it by minified json
Environment variables
- MODELZ_CTRL_HOST
Provide a default for
--host
- MODELZ_USER
Provide a default for
--user-id
- MODELZ_API_KEY
Provide a default for
--key
delete¶
deployment delete [OPTIONS]
Options
- --host <host>¶
Control Apiserver host for Modelz, will read from env $MODELZ_CTRL_HOST if not provided
- -u, --user-id <user_id>¶
Required login name for Modelz, will read from env $MODELZ_USER if not provided
- -k, --key <key>¶
Required API key for Modelz, will read from env $MODELZ_API_KEY if not provided
- -d, --deployment <deployment>¶
Required Deployment id
Environment variables
- MODELZ_CTRL_HOST
Provide a default for
--host
- MODELZ_USER
Provide a default for
--user-id
- MODELZ_API_KEY
Provide a default for
--key
get¶
deployment get [OPTIONS]
Options
- --host <host>¶
Control Apiserver host for Modelz, will read from env $MODELZ_CTRL_HOST if not provided
- -u, --user-id <user_id>¶
Required login name for Modelz, will read from env $MODELZ_USER if not provided
- -k, --key <key>¶
Required API key for Modelz, will read from env $MODELZ_API_KEY if not provided
- -d, --deployment <deployment>¶
Required Deployment id
Environment variables
- MODELZ_CTRL_HOST
Provide a default for
--host
- MODELZ_USER
Provide a default for
--user-id
- MODELZ_API_KEY
Provide a default for
--key
list¶
deployment list [OPTIONS]
Options
- --host <host>¶
Control Apiserver host for Modelz, will read from env $MODELZ_CTRL_HOST if not provided
- -u, --user-id <user_id>¶
Required login name for Modelz, will read from env $MODELZ_USER if not provided
- -k, --key <key>¶
Required API key for Modelz, will read from env $MODELZ_API_KEY if not provided
Environment variables
- MODELZ_CTRL_HOST
Provide a default for
--host
- MODELZ_USER
Provide a default for
--user-id
- MODELZ_API_KEY
Provide a default for
--key
update¶
deployment update [OPTIONS]
Options
- --host <host>¶
Control Apiserver host for Modelz, will read from env $MODELZ_CTRL_HOST if not provided
- -u, --user-id <user_id>¶
Required login name for Modelz, will read from env $MODELZ_USER if not provided
- -k, --key <key>¶
Required API key for Modelz, will read from env $MODELZ_API_KEY if not provided
- -d, --deployment <deployment>¶
Required Deployment id
- --min-replicas <min_replicas>¶
MinReplicas is the minimum number of replicas of the deployment
- --max-replicas <max_replicas>¶
MaxReplicas is the maximum number of replicas of the deployment
- --target-load <target_load>¶
TargetLoad is the target load of the deployment. (inflight requests per replica)
- --startup-duration <startup_duration>¶
StartupDuration is the startup timeout
- --zero-duration <zero_duration>¶
ZeroDuration is the idle timeout before scaling to zero
- --env-vars <env_vars>¶
EnvVars is the environment variables of the deployment, input it by minified json
Environment variables
- MODELZ_CTRL_HOST
Provide a default for
--host
- MODELZ_USER
Provide a default for
--user-id
- MODELZ_API_KEY
Provide a default for
--key