llama（客户端）CLI 参考

llama-stack-client CLI 允许您查询关于分发版的信息。

基本命令

`llama-stack-client`

llama-stack-client
Usage: llama-stack-client [OPTIONS] COMMAND [ARGS]...

  Welcome to the LlamaStackClient CLI

Options:
  --version        Show the version and exit.
  --endpoint TEXT  Llama Stack distribution endpoint
  --api-key TEXT   Llama Stack distribution API key
  --config TEXT    Path to config file
  --help           Show this message and exit.

Commands:
  configure          Configure Llama Stack Client CLI.
  datasets           Manage datasets.
  eval               Run evaluation tasks.
  eval_tasks         Manage evaluation tasks.
  inference          Inference (chat).
  inspect            Inspect server configuration.
  models             Manage GenAI models.
  post_training      Post-training.
  providers          Manage API providers.
  scoring_functions  Manage scoring functions.
  shields            Manage safety shield services.
  toolgroups         Manage available tool groups.
  vector_dbs         Manage vector databases.

`llama-stack-client configure`

llama-stack-client configure
> Enter the host name of the Llama Stack distribution server: localhost
> Enter the port number of the Llama Stack distribution server: 8321
Done! You can now use the Llama Stack Client CLI with endpoint https://:8321

`llama-stack-client providers list`

llama-stack-client providers list

+-----------+----------------+-----------------+
| API       | Provider ID    | Provider Type   |
+===========+================+=================+
| scoring   | meta0          | meta-reference  |
+-----------+----------------+-----------------+
| datasetio | meta0          | meta-reference  |
+-----------+----------------+-----------------+
| inference | tgi0           | remote::tgi     |
+-----------+----------------+-----------------+
| memory    | meta-reference | meta-reference  |
+-----------+----------------+-----------------+
| agents    | meta-reference | meta-reference  |
+-----------+----------------+-----------------+
| telemetry | meta-reference | meta-reference  |
+-----------+----------------+-----------------+
| safety    | meta-reference | meta-reference  |
+-----------+----------------+-----------------+

模型管理

`llama-stack-client models list`

llama-stack-client models list

Available Models

┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ model_type   ┃ identifier                           ┃ provider_resource_id         ┃ metadata  ┃ provider_id ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ llm          │ meta-llama/Llama-3.2-3B-Instruct     │ llama3.2:3b-instruct-fp16    │           │ ollama      │
└──────────────┴──────────────────────────────────────┴──────────────────────────────┴───────────┴─────────────┘

Total models: 1

`llama-stack-client models get`

llama-stack-client models get Llama3.1-8B-Instruct

+----------------------+----------------------+----------------------------------------------------------+---------------+
| identifier           | llama_model          | metadata                                                 | provider_id   |
+======================+======================+==========================================================+===============+
| Llama3.1-8B-Instruct | Llama3.1-8B-Instruct | {'huggingface_repo': 'meta-llama/Llama-3.1-8B-Instruct'} | tgi0          |
+----------------------+----------------------+----------------------------------------------------------+---------------+

llama-stack-client models get Random-Model

Model RandomModel is not found at distribution endpoint host:port. Please ensure endpoint is serving specified model.

`llama-stack-client models register`

llama-stack-client models register <model_id> [--provider-id <provider_id>] [--provider-model-id <provider_model_id>] [--metadata <metadata>]

`llama-stack-client models update`

llama-stack-client models update <model_id> [--provider-id <provider_id>] [--provider-model-id <provider_model_id>] [--metadata <metadata>]

`llama-stack-client models delete`

llama-stack-client models delete <model_id>

向量数据库管理

`llama-stack-client vector_dbs list`

llama-stack-client vector_dbs list

+--------------+----------------+---------------------+---------------+------------------------+
| identifier   | provider_id    | provider_resource_id| vector_db_type| params                |
+==============+================+=====================+===============+========================+
| test_bank    | meta-reference | test_bank          | vector        | embedding_model: all-MiniLM-L6-v2
                                                                      embedding_dimension: 384|
+--------------+----------------+---------------------+---------------+------------------------+

`llama-stack-client vector_dbs register`

llama-stack-client vector_dbs register <vector-db-id> [--provider-id <provider-id>] [--provider-vector-db-id <provider-vector-db-id>] [--embedding-model <embedding-model>] [--embedding-dimension <embedding-dimension>]

可选参数

--provider-id: 向量数据库的提供者 ID
--provider-vector-db-id: 提供者的向量数据库 ID
--embedding-model: 要使用的嵌入模型。默认值：“all-MiniLM-L6-v2”
--embedding-dimension: 嵌入的维度。默认值：384

`llama-stack-client vector_dbs unregister`

llama-stack-client vector_dbs unregister <vector-db-id>

屏蔽管理

`llama-stack-client shields list`

llama-stack-client shields list

+--------------+----------+----------------+-------------+
| identifier   | params   | provider_id    | type        |
+==============+==========+================+=============+
| llama_guard  | {}       | meta-reference | llama_guard |
+--------------+----------+----------------+-------------+

`llama-stack-client shields register`

llama-stack-client shields register --shield-id <shield-id> [--provider-id <provider-id>] [--provider-shield-id <provider-shield-id>] [--params <params>]

必需参数

--shield-id: 屏蔽的 ID

可选参数

--provider-id: 屏蔽的提供者 ID
--provider-shield-id: 提供者的屏蔽 ID
--params: 屏蔽的 JSON 配置参数

评估任务管理

`llama-stack-client benchmarks list`

llama-stack-client benchmarks list

`llama-stack-client benchmarks register`

llama-stack-client benchmarks register --eval-task-id <eval-task-id> --dataset-id <dataset-id> --scoring-functions <function1> [<function2> ...] [--provider-id <provider-id>] [--provider-eval-task-id <provider-eval-task-id>] [--metadata <metadata>]

必需参数

--eval-task-id: 评估任务的 ID
--dataset-id: 要评估的数据集的 ID
--scoring-functions: 一个或多个用于评估的评分函数

可选参数

--provider-id: 评估任务的提供者 ID
--provider-eval-task-id: 提供者的评估任务 ID
--metadata: JSON 格式的评估任务元数据

评估执行

`llama-stack-client eval run-benchmark`

llama-stack-client eval run-benchmark <eval-task-id1> [<eval-task-id2> ...] --eval-task-config <config-file> --output-dir <output-dir> [--num-examples <num>] [--visualize]

必需参数

--eval-task-config: JSON 格式的评估任务配置文件路径
--output-dir: 保存评估结果的目录路径

可选参数

--num-examples: 要评估的样本数量（对调试有用）
--visualize: 如果设置，在评估完成后可视化评估结果

示例 benchmark_config.json

{
    "type": "benchmark",
    "eval_candidate": {
        "type": "model",
        "model": "Llama3.1-405B-Instruct",
        "sampling_params": {
            "strategy": "greedy",
        }
    }
}

`llama-stack-client eval run-scoring`

llama-stack-client eval run-scoring <eval-task-id> --eval-task-config <config-file> --output-dir <output-dir> [--num-examples <num>] [--visualize]

必需参数

--eval-task-config: JSON 格式的评估任务配置文件路径
--output-dir: 保存评分结果的目录路径

可选参数

--num-examples: 要评估的样本数量（对调试有用）
--visualize: 如果设置，在评分完成后可视化评分结果

工具组管理

`llama-stack-client toolgroups list`

llama-stack-client toolgroups list

+---------------------------+------------------+------+---------------+
| identifier                | provider_id      | args | mcp_endpoint  |
+===========================+==================+======+===============+
| builtin::rag              | rag-runtime      | None | None          |
+---------------------------+------------------+------+---------------+
| builtin::websearch        | tavily-search    | None | None          |
+---------------------------+------------------+------+---------------+

`llama-stack-client toolgroups get`

llama-stack-client toolgroups get <toolgroup_id>

显示特定工具组的详细信息。如果未找到该工具组，则显示错误消息。

`llama-stack-client toolgroups register`

llama-stack-client toolgroups register <toolgroup_id> [--provider-id <provider-id>] [--provider-toolgroup-id <provider-toolgroup-id>] [--mcp-config <mcp-config>] [--args <args>]

可选参数

--provider-id: 工具组的提供者 ID
--provider-toolgroup-id: 提供者的工具组 ID
--mcp-config: MCP 端点的 JSON 配置
--args: 工具组的 JSON 参数

`llama-stack-client toolgroups unregister`

llama-stack-client toolgroups unregister <toolgroup_id>

llama（客户端）CLI 参考

基本命令

llama-stack-client

llama-stack-client configure

llama-stack-client providers list

模型管理

llama-stack-client models list

llama-stack-client models get

llama-stack-client models register

llama-stack-client models update

llama-stack-client models delete

向量数据库管理

llama-stack-client vector_dbs list

llama-stack-client vector_dbs register

llama-stack-client vector_dbs unregister

屏蔽管理

llama-stack-client shields list

llama-stack-client shields register

评估任务管理

llama-stack-client benchmarks list

llama-stack-client benchmarks register

评估执行

llama-stack-client eval run-benchmark

llama-stack-client eval run-scoring

工具组管理

llama-stack-client toolgroups list

llama-stack-client toolgroups get

llama-stack-client toolgroups register

llama-stack-client toolgroups unregister

`llama-stack-client`

`llama-stack-client configure`

`llama-stack-client providers list`

`llama-stack-client models list`

`llama-stack-client models get`

`llama-stack-client models register`

`llama-stack-client models update`

`llama-stack-client models delete`

`llama-stack-client vector_dbs list`

`llama-stack-client vector_dbs register`

`llama-stack-client vector_dbs unregister`

`llama-stack-client shields list`

`llama-stack-client shields register`

`llama-stack-client benchmarks list`

`llama-stack-client benchmarks register`

`llama-stack-client eval run-benchmark`

`llama-stack-client eval run-scoring`

`llama-stack-client toolgroups list`

`llama-stack-client toolgroups get`

`llama-stack-client toolgroups register`

`llama-stack-client toolgroups unregister`