Skip to content

Commit

Permalink
feat(llm-inference): add support llm-inference cli (#3731)
Browse files Browse the repository at this point in the history
  • Loading branch information
Laure-di authored Apr 2, 2024
1 parent befc87f commit 7d2386f
Show file tree
Hide file tree
Showing 44 changed files with 1,814 additions and 4 deletions.
21 changes: 21 additions & 0 deletions cmd/scw/testdata/test-all-usage-llm-inference-acl-add-usage.golden
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Add new ACL rules for a specific deployment.

USAGE:
scw llm-inference acl add [arg=value ...]

ARGS:
deployment-id ID of the deployment to add ACL rules to
[acls.{index}.ip] IP address to be allowed
[acls.{index}.description] Description of the ACL rule
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par)

FLAGS:
-h, --help help for add

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Delete an exising ACL.

USAGE:
scw llm-inference acl delete [arg=value ...]

ARGS:
acl-id ID of the ACL rule to delete
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par)

FLAGS:
-h, --help help for delete

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
List ACLs for a specific deployment.

USAGE:
scw llm-inference acl list [arg=value ...]

ARGS:
deployment-id ID of the deployment to list ACL rules for
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par | all)

FLAGS:
-h, --help help for list

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
21 changes: 21 additions & 0 deletions cmd/scw/testdata/test-all-usage-llm-inference-acl-set-usage.golden
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Set new ACL rules for a specific deployment.

USAGE:
scw llm-inference acl set [arg=value ...]

ARGS:
deployment-id ID of the deployment to set ACL rules for
[acls.{index}.ip] IP address to be allowed
[acls.{index}.description] Description of the ACL rule
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par)

FLAGS:
-h, --help help for set

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
23 changes: 23 additions & 0 deletions cmd/scw/testdata/test-all-usage-llm-inference-acl-usage.golden
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Access Control List (ACL) management commands.

USAGE:
scw llm-inference acl <command>

AVAILABLE COMMANDS:
add Add new ACLs
delete Delete an exising ACL
list List your ACLs
set Set new ACL

FLAGS:
-h, --help help for acl

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use

Use "scw llm-inference acl [command] --help" for more information about a command.
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Create a new inference deployment related to a specific model.

USAGE:
scw llm-inference deployment create [arg=value ...]

ARGS:
name=<generated> Name of the deployment
[project-id] Project ID to use. If none is passed the default project ID will be used
model-name Name of the model to use
[accept-eula] Accept the model's End User License Agreement (EULA).
node-type Name of the node type to use
[tags.{index}] List of tags to apply to the deployment
[min-size] Defines the minimum size of the pool
[max-size] Defines the maximum size of the pool
[endpoints.{index}.is-public=false] Will configure your public endpoint if true
[endpoints.{index}.private-network.private-network-id] ID of the Private Network
[endpoints.{index}.disable-auth=false] Disable the authentication on the endpoint.
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par)

FLAGS:
-h, --help help for create
-w, --wait wait until the deployment is ready

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Delete an existing inference deployment.

USAGE:
scw llm-inference deployment delete <deployment-id ...> [arg=value ...]

ARGS:
deployment-id ID of the deployment to delete
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par)

FLAGS:
-h, --help help for delete
-w, --wait wait until the deployment is ready

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Get the CA certificate used for the deployment of private endpoints.
The CA certificate will be returned as a PEM file.

USAGE:
scw llm-inference deployment get-certificate [arg=value ...]

ARGS:
deployment-id
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par)

FLAGS:
-h, --help help for get-certificate

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Get the deployment for the given ID.

USAGE:
scw llm-inference deployment get <deployment-id ...> [arg=value ...]

ARGS:
deployment-id ID of the deployment to get
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par)

FLAGS:
-h, --help help for get

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
List all your inference deployments.

USAGE:
scw llm-inference deployment list [arg=value ...]

ARGS:
[order-by] Order in which to return results (created_at_desc | created_at_asc | name_asc | name_desc)
[project-id] Filter by Project ID
[name] Filter by deployment name
[tags.{index}] Filter by tags
[organization-id] Filter by Organization ID
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par | all)

FLAGS:
-h, --help help for list

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Update an existing inference deployment.

USAGE:
scw llm-inference deployment update <deployment-id ...> [arg=value ...]

ARGS:
deployment-id ID of the deployment to update
[name] Name of the deployment
[tags.{index}] List of tags to apply to the deployment
[min-size] Defines the new minimum size of the pool
[max-size] Defines the new maximum size of the pool
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par)

FLAGS:
-h, --help help for update

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Deployment commands.

USAGE:
scw llm-inference deployment <command>

AVAILABLE COMMANDS:
create Create a deployment
delete Delete a deployment
get Get a deployment
get-certificate Get the CA certificate
list List inference deployments
update Update a deployment

FLAGS:
-h, --help help for deployment

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use

Use "scw llm-inference deployment [command] --help" for more information about a command.
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Create a new Endpoint related to a specific deployment.

USAGE:
scw llm-inference endpoint create [arg=value ...]

ARGS:
deployment-id ID of the deployment to create the endpoint for
[endpoint.is-public=false] Will configure your public endpoint if true
[endpoint.private-network.private-network-id] ID of the Private Network
[endpoint.disable-auth=false] Disable the authentication on the endpoint.
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par)

FLAGS:
-h, --help help for create

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Delete an existing Endpoint.

USAGE:
scw llm-inference endpoint delete <endpoint-id ...> [arg=value ...]

ARGS:
endpoint-id ID of the endpoint to delete
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par)

FLAGS:
-h, --help help for delete

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Update an existing Endpoint.

USAGE:
scw llm-inference endpoint update <endpoint-id ...> [arg=value ...]

ARGS:
endpoint-id ID of the endpoint to update
[disable-auth] Disable the authentication on the endpoint.
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par)

FLAGS:
-h, --help help for update

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Endpoint management commands.

USAGE:
scw llm-inference endpoint <command>

AVAILABLE COMMANDS:
create Create an endpoint
delete Delete an endpoint
update Update an endpoint

FLAGS:
-h, --help help for endpoint

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use

Use "scw llm-inference endpoint [command] --help" for more information about a command.
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
🎲🎲🎲 EXIT CODE: 0 🎲🎲🎲
🟥🟥🟥 STDERR️️ 🟥🟥🟥️
Get the model for the given ID.

USAGE:
scw llm-inference model get <model-id ...> [arg=value ...]

ARGS:
model-id ID of the model to get
[region=fr-par] Region to target. If none is passed will use default region from the config (fr-par)

FLAGS:
-h, --help help for get

GLOBAL FLAGS:
-c, --config string The path to the config file
-D, --debug Enable debug mode
-o, --output string Output format: json or human, see 'scw help output' for more info (default "human")
-p, --profile string The config profile to use
Loading

0 comments on commit 7d2386f

Please sign in to comment.