Modelarts Service (modelarts)¶

The Modelarts client is the command-line interface (CLI) for the ModelArts service API and its extensions.

For help on a specific rds command, enter:

$ openstack ma help SUBCOMMAND

Built-In Model Operations¶

ma builtin model list¶

Get properties of a vm

openstack ma builtin model list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--limit <limit>]
    [--offset <offset>]
    [--sort-by {create_time,engine,model_name,model_precision,model_usage,model_precision,model_sizeparameter}]
    [--order {asc, desc}]
    [--search-content <search_content>]

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--limit <limit>¶: Number of built-in algorithms displayed on each page. The value range is [1, 100]. Default value: 10.

--offset <offset>¶: Index of the page to be queried. Default value: 1

--sort-by {create_time,engine,model_name,model_precision,model_usage,model_precision,model_sizeparameter}¶: Sorting field. Default value: job_name

--order {asc, desc}¶: Sorting order. Default value: desc

--search-content <search_content>¶: Search content, for example, a parameter name.

This command is provided by the otcextensions plugin.

ma builtin model show¶

Show details of a built-in model

openstack ma builtin model show <name>

name¶: Enter model built-in model name.

This command is provided by the otcextensions plugin.

Devenv Operations¶

ma devenv create¶

Create a Devenv Instance.

openstack ma devenv create
    --profile-id <profile_id>
    [--description <description>]
    --flavor <flavor>
    [--ai-project-id <ai_project_id>]
    [--workspace-id <workspace_id>]
    --storage-type <storage_type>
    --storage-path <storage_path>
    [--extended-storage-type <extended_storage_type>]
    [--extended-storage-path <extended_storage_path>]
    [--autostop-duration <autostop_duration>]
    [--autostop-prompt <autostop_prompt>]
    [--pool-id <pool_id>]
    [--pool-type <pool_type>]
    [--pool-name <pool_name>]
    [--annotation <annotation>]
    <name>

--profile-id <profile_id>¶: Configuration ID. The options are as follows: 1: Multi-Engine 1.0 (python3)-cpu 2: Multi-Engine 1.0 (python3)-gpu 3: Multi-Engine 2.0 (python3)

--description <description>¶: Instance description (max 512 characters.)

--flavor <flavor>¶: Instance flavor.

--ai-project-id <ai_project_id>¶: AI project ID. This parameter is reserved.

--workspace-id <workspace_id>¶: Workspace. The default workspace ID is 0.

--storage-type <storage_type>¶: Storage type. Only obs is supported.

--storage-path <storage_path>¶: The value must be a valid OBS bucket path and end with a slash (/). The value must be a specific directory in an OBS bucket rather than the root directory of an OBS bucket.

--extended-storage-type <extended_storage_type>¶: Storage type. Only obs is supported.

--extended-storage-path <extended_storage_path>¶: If extended_storage_type is set to obs, this parameter is mandatory. The value must be a valid OBS bucket path and end with a slash (/). The value must be a specific directory in an OBS bucket rather than the root directory of an OBS bucket.

--autostop-duration <autostop_duration>¶: Running duration, in seconds. The value ranges from 3600 to 86400. The instance will automatically stop when the running duration is reached. After this parameter is set, it is valid for each startup.

--autostop-prompt <autostop_prompt>¶: Whether to display a prompt again. This parameter is provided for the console to determine whether to display a prompt again. The default value is true.

--pool-id <pool_id>¶: Resource pool ID.

--pool-type <pool_type>¶: Resource pool Type.

--pool-name <pool_name>¶: Resource pool Name.

--annotation <annotation>¶: Custom data to be passed as Map<String:String>. Data may be passed as <key>=<value> or JSON. (repeat option to set multiple annotations)

name¶: Instance name (max 64 characters). The value can contain letters, digits, hyphens (-), and underscores (_).

This command is provided by the otcextensions plugin.

ma devenv delete¶

Delete Devenv Instance(s)

openstack ma devenv delete <instance> [<instance> ...]

instance¶: ID or Name of the Devenv Instance(s) to be deleted.

This command is provided by the otcextensions plugin.

ma devenv list¶

List Devenv Instances.

openstack ma devenv list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--dev-env-type <dev_env_type>]
    [--provision-type PROVISION_TYPE]
    [--status <status>]
    [--sort-by <sort_by>]
    [--order {asc,desc}]
    [--offset <offset>]
    [--limit <limit>]
    [--workspace-id <workspace_id>]
    [--ai-project-id <ai_project_id>]
    [--pool-id <pool_id>]
    [--show-self]

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--dev-env-type <dev_env_type>¶: Development environment type. Only Notebook is supported. The first letter must be capitalized.

--provision-type <PROVISION_TYPE>¶: Deployment type. Only Docker is supported.

--status <status>¶: Filter Devenv Instances by status.

--sort-by <sort_by>¶: Classification standard. The value can be name or creation_timestamp. By default list is sorted by name.

--order {asc,desc}¶: Sorting order. Default value: asc

--offset <offset>¶: Start index. The default value is 0.

--limit <limit>¶: Number of returned result records.

--workspace-id <workspace_id>¶: Workspace ID. If no workspace is created, the default value is 0.

--ai-project-id <ai_project_id>¶: AI project ID.

--pool-id <pool_id>¶: ID of a dedicated resource pool.

--show-self¶: Only the current user’s instanaces will be listed.

This command is provided by the otcextensions plugin.

ma devenv show¶

Show details of a Devenv Instance.

openstack ma devenv show <instance>

instance¶: Devenv Instance ID.

This command is provided by the otcextensions plugin.

ma devenv start¶

Start a Devenv Instance.

openstack ma devenv start <instance>

instance¶: Devenv Instance name or ID.

This command is provided by the otcextensions plugin.

ma devenv stop¶

Start a Devenv Instance.

openstack ma devenv stop <instance>

instance¶: Devenv Instance name or ID.

This command is provided by the otcextensions plugin.

ma devenv update¶

Update Devenv Description

openstack ma devenv update
    [--description <description>]
    [--auto-stop {enable,disable}]
    [--duration <duration>]
    [--prompt {enable,disable}]
    <instance>

--description <description>¶: Additional service attribute, which facilitates service management.

--auto-stop {enable,disable}¶: Whether to enable or disable the auto stop function.

--duration <duration>¶: Running duration, in seconds. The value ranges from 3,600 to 86,400. This parameter is mandatory when auto_stop is enabled.

--prompt {enable,disable}¶: Whether to disable a display prompt again for auto_stop. This parameter is provided for the console to determine whether to display a prompt again.

instance¶: Devenv Instance ID.

This command is provided by the otcextensions plugin.

Model Operations¶

ma model create¶

Create a ModelArts model

openstack ma model create
    --model-version <model_version>
    --source-location <source_location>
    [--source-job-id <source_job_id>]
    [--source-job-version <source_job_version>]
    [--source-type <source_type>]
    --model-type <model_type>
    [--runtime <runtime>]
    [--description <description>]
    [--execution-code <execution_code>]
    [--input-params <input_params>]
    [--output-params <output_params>]
    [--dependencies <dependencies>]
    [--model-algorithm <model_algorithm>]
    [--model-metrics <model_metrics>]
    [--apis <apis>]
    [--initial-config <initial_config>]
    [--workspace-id <workspace_id>]
    [--model-docs <model_docs>]
    [--install-type <install_type>]
    <name>

--model-version <model_version>¶: Model version in the format of Digit.Digit.Digit. The value range of the digits is [1, 99]. Note that no part of the version number can start with 0. For example, 01.01.01 is not allowed.

--source-location <source_location>¶: OBS path where the model is located or the template address of the SWR image.

--source-job-id <source_job_id>¶: ID of the source training job. If the model is generated from a training job, input this parameter for source tracing. If the model is imported from a third-party meta model, leave this parameter blank. By default, this parameter is left blank.

--source-job-version <source_job_version>¶: Version of the source training job. If the model is generated from a training job, input this parameter for source tracing. If the model is imported from a third-party meta model, leave this parameter blank. By default, this parameter is left blank.

--source-type <source_type>¶: Model source type. Currently, the value can only be auto, which indicates ExeML models (model download is not supported). If the model is deployed by a training job, leave this parameter blank. By default, this parameter is left blank.

--model-type <model_type>¶: Model type. The value can be TensorFlow, MXNet, Caffe, Spark_MLlib, Scikit_Learn, XGBoost, Image, or PyTorch, which is read from the configuration file.

--runtime <runtime>¶: Model running environment. The possible values of runtime are related to model_type.

--description <description>¶: Model remarks. The value contains a maximum of 100 characters and cannot contain the following special characters and more: &!’”<>=

--execution-code <execution_code>¶: OBS path for storing the execution code. By default, this parameter is left blank. The name of the execution code file is fixed to customize_service.py.

--input-params <input_params>¶: Collection of input parameters of a model. By default, this parameter is left blank.

--output-params <output_params>¶: Collection of output parameters of a model. By default, this parameter is left blank.

--dependencies <dependencies>¶: Package required for inference code and model. By default, this parameter is left blank.

--model-algorithm <model_algorithm>¶: Model algorithm. If the algorithm is read from the configuration file, this parameter can be left blank. For example, the value can be predict_analysis, object_detection, or image_classification.

--model-metrics <model_metrics>¶: Model precision, which is read from the configuration file.

--apis <apis>¶: All apis input and output parameters of the model. If the parameters are read from the configuration file, this parameter can be left blank.

--initial-config <initial_config>¶: Character string converted from the final model configuration file. It is recommended that the initial_config file be used to provide information about the fields such as apis, dependencies, input_params, and output_params.

--workspace-id <workspace_id>¶: Workspace ID. Default value: 0

--model-docs <model_docs>¶: List of model description documents. A maximum of three documents are supported.

--install-type <install_type>¶: Deployment type. Only lowercase letters are supported. The value can be real-time, or batch. Default value: [“real-time”,”batch”]

name¶: Model name. The value can contain 1 to 64 visible characters, including Chinese characters. Only letters, Chinese characters, digits, hyphens (-), and underscores (_) are allowed.

This command is provided by the otcextensions plugin.

ma model delete¶

Delete ModelArts Model(s)

openstack ma model delete <model> [<model> ...]

model¶: ID or Name of the ModelArts model(s) to be deleted.

This command is provided by the otcextensions plugin.

ma model list¶

List models

openstack ma model list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--limit <limit>]
    [--offset <offset>]
    [--order <order>]
    [--workspace-id <workspace_id>]
    [--sort-by <sort_by>]
    [--name <name>]
    [--model-version <model_version>]
    [--status <status>]
    [--model-type <model_type>]
    [--not-model-type <not_model_type>]
    [--description <description>]

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--limit <limit>¶: Maximum number of records returned on each page. The default value is 100. The recommended value ranges from 10 to 50.

--offset <offset>¶: Index of the page to be queried. Default value: 0.

--order <order>¶: Sorting order. The value can be asc or desc, indicating ascending or descending order. Default value: desc

--workspace-id <workspace_id>¶: ID of the workspace to which a service belongs. The default value is 0, indicating the default workspace.

--sort-by <sort_by>¶: Sorting mode. The value can be create_at, model_version, or model_size. Default value: create_at.

--name <name>¶: Model name. Fuzzy match is supported.

--model-version <model_version>¶: Model version.

--status <status>¶: Model status.

--model-type <model_type>¶: Model type. The models of this type are queried. model_type and not_model_type are mutually exclusive and cannot co-exist.

--not-model-type <not_model_type>¶: Model type. A list of models of types except for this type are queried.

--description <description>¶: Description. Fuzzy match is supported.

This command is provided by the otcextensions plugin.

ma model show¶

Show details of a modelartsv1 model

openstack ma model show <model>

model¶: Enter model id or name

This command is provided by the otcextensions plugin.

Service Operations¶

ma service cluster list¶

List dedicated clusters for service deployment.

openstack ma service cluster list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--cluster-name <cluster_name>]
    [--status <status>]
    [--offset <offset>]
    [--limit <limit>]
    [--sort-by <sort_by>]
    [--order <order>]

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--cluster-name <cluster_name>¶: Cluster name.

--status <status>¶: Cluster Status.

--offset <offset>¶: Start page of the paging list. Default value: 0

--limit <limit>¶: Maximum number of records returned on each page. Default value: 1000

--sort-by <sort_by>¶: Sorting field. The options are as follows: created_at: default value cluster_name

--order <order>¶: Sorting mode. The default value is desc. Options: asc: ascending order desc: descending order

This command is provided by the otcextensions plugin.

ma service create¶

Create a ModelArts service

openstack ma service create
    [--description <description>]
    --infer-type
    {real-time,batch}
    [--router-id <router_id>]
    [--network-id <network_id>]
    [--security-group-id <security_group_id>]
    [--cluster-id <cluster_id>]
    [--schedule type=<type>,time_unit=<time_unit>,duration=<duration>]
    [--workspace-id <workspace_id>]
    --model-id <model_id>
    [--weight <weight>]
    --specification <specification>
    --instance-count <instance_count>
    [--env <key=value>]
    [--custom-spec cpu=<cpu>,memory=<memory>,gpu_p4=<gpu_p4>]
    [--src-type <src_type>]
    [--src-path <src_path>]
    [--dest-path <dest_path>]
    [--req-uri <dest_path>]
    [--mapping-type {file, csv}]
    [--mapping-rule <mapping_rule>]
    [--wait]
    [--timeout <timeout>]
    <name>

--description <description>¶: Service description, which contains a maximum of 100 characters. By default, this parameter is left blank.

--infer-type {real-time,batch}¶: Inference mode.

--router-id <router_id>¶: ID of the VPC to which a real-time service instance is deployed. By default, this parameter is left blank.

--network-id <network_id>¶: ID of a subnet. By default, this parameter is left blank. This parameter is mandatory when vpc_id is configured. Enter the network ID displayed in the subnet details on the VPC management console

--security-group-id <security_group_id>¶: A security group is a virtual firewall that provides secure network access control policies for service instances.

--cluster-id <cluster_id>¶: ID of a dedicated resource pool. If this parameter is set –router-id doesnt take effect.

--schedule type=<type>,time_unit=<time_unit>,duration=<duration>¶: Service scheduling configuration, which can be configured only for real-time services.

--workspace-id <workspace_id>¶: Workspace ID.

--model-id <model_id>¶: Model ID

--weight <weight>¶: Traffic weight allocated to a model. This parameter is mandatory only when infer_type is set to real-time. The sum of the weights must be 100.

--specification <specification>¶: Resource specifications. Select specifications based on service requirements.

--instance-count <instance_count>¶: Number of instances deployed in a model The value must be greater than 0.

--env <key=value>¶: Environment variable key-value pair required for running a model. Example: –env VAR1=value1 –env VAR2=value2

--custom-spec cpu=<cpu>,memory=<memory>,gpu_p4=<gpu_p4>¶: Custom specifications. Set this parameter when you use a dedicated resource pool.

--src-type <src_type>¶: Data source type.

--src-path <src_path>¶: OBS path of the input data of a batch job.

--dest-path <dest_path>¶: OBS path of the output data of a batch job.

--req-uri <dest_path>¶: API URI from the model config.json file for inference. If a ModelArts built-in inference image is used, the value of this parameter is /

--mapping-type {file, csv}¶: Mapping type of the input data.

--mapping-rule <mapping_rule>¶: Mapping between input parameters and CSV data. This parameter is mandatory only when mapping_type is set to csv.

--wait¶: Wait for service deployment.

--timeout <timeout>¶: Timeout for the wait in seconds (default 1200 seconds).

name¶: Service name. Enter 1 to 64 characters. Only letters, digits, hyphens (-), and underscores (_) are allowed.

This command is provided by the otcextensions plugin.

ma service delete¶

Delete ModelArts Service(s)

openstack ma service delete <service> [<service> ...]

service¶: ID or Name of the ModelArts service(s) to be deleted.

This command is provided by the otcextensions plugin.

ma service events¶

List update logs of a real-time service.

openstack ma service events
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--event-type <event_type>]
    [--start-time <start_time>]
    [--end-time <end_time>]
    [--offset <offset>]
    [--limit <limit>]
    [--sort-by <sort_by>]
    [--order <order>]
    <service>

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--event-type <event_type>¶: Type of the event to be filtered. By default, the event type is not filtered. Options: normal: normal events abnormal: abnormal events

--start-time <start_time>¶: Start time of the event to be filtered. The value is milliseconds.

--end-time <end_time>¶: End time of the event to be filtered. The value is milliseconds.

--offset <offset>¶: Start page of the paging list. Default value: 0

--limit <limit>¶: Maximum number of records returned on each page. Default value: 1000

--sort-by <sort_by>¶: Specified sorting field. The default value is occur_time.

--order <order>¶: Sorting mode. The default value is desc. Options: asc: ascending order desc: descending order

service¶: Service ID or Name.

This command is provided by the otcextensions plugin.

ma service flavor list¶

List Service Deployment Specifications.

openstack ma service flavor list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--is-personal-cluster]
    [--infer-type {real-time,batch}]

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--is-personal-cluster¶: Whether to query the service deployment specifications supported by dedicated resource pool. The default value is false.

--infer-type {real-time,batch}¶: Inference mode. The default value is real-time. The value can be real-time or batch.

This command is provided by the otcextensions plugin.

ma service list¶

Get properties of a service

openstack ma service list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--cluster-id <cluster_id>]
    [--infer-type {real-time,batch}]
    [--limit <limit>]
    [--model-id <model_id>]
    [--offset <offset>]
    [--order {asc, desc}]
    [--service-id <service_id>]
    [--name <name>]
    [--sort-by {publish_at, service_name}]
    [--status {running,deploying,concerning,failed,stopped,finished}]
    [--workspace-id <workspace_id>]

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--cluster-id <cluster_id>¶: Dedicated resource pool ID.

--infer-type {real-time,batch}¶: Inference mode.

--limit <limit>¶: Records Limit. Default value: 1000

--model-id <model_id>¶: Model Name.

--offset <offset>¶: Start page of the paging list. Default value: 0

--order {asc, desc}¶: Sorting order. Default value: desc

--service-id <service_id>¶: Service ID.

--name <name>¶: Service Name.

--sort-by {publish_at, service_name}¶: Sorting field. Default value: publish_at

--status {running,deploying,concerning,failed,stopped,finished}¶: Service status.

--workspace-id <workspace_id>¶: Workspace ID.

This command is provided by the otcextensions plugin.

ma service logs¶

List update logs of a real-time service.

openstack ma service logs
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--update-time <update_time>]
    <service>

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--update-time <update_time>¶: Update time for filtering. This parameter can be used to obtain the update logs of a real-time service. By default, the filtering by update time is disabled.

service¶: Service ID or Name.

This command is provided by the otcextensions plugin.

ma service monitor¶

Get Monitoring Information of a Service.

openstack ma service monitor
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--node-id <node_id>]
    <service>

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--node-id <node_id>¶: ID of the node to be queried. By default, all nodes are queried.

service¶: Service ID or Name.

This command is provided by the otcextensions plugin.

ma service show¶

Show details of a Modelarts serivce

openstack ma service show <service>

service¶: Service Name or ID

This command is provided by the otcextensions plugin.

ma service start¶

Start a Service.

openstack ma service start <service>

service¶: Service name or ID.

This command is provided by the otcextensions plugin.

ma service stop¶

Stop a Service.

openstack ma service stop <service>

service¶: Service name or ID.

This command is provided by the otcextensions plugin.

ma service update¶

Update Service Configuration

openstack ma service update
    [--description <description>]
    [--schedule type=<type>,time_unit=<time_unit>,duration=<duration>]
    [--additional-property <key=value>]
    [--model-id <model_id>]
    [--weight <weight>]
    [--cluster-id <cluster_id>]
    [--specification <specification>]
    [--instance-count <instance_count>]
    [--env <key=value>]
    [--custom-spec cpu=<cpu>,memory=<memory>,gpu_p4=<gpu_p4>]
    [--src-type <src_type>]
    [--src-path <src_path>]
    [--dest-path <dest_path>]
    [--req-uri <dest_path>]
    [--mapping-type {file, csv}]
    [--mapping-rule <mapping_rule>]
    [--wait]
    [--timeout <timeout>]
    <service>

--description <description>¶: Service description (max 100 characters).

--schedule type=<type>,time_unit=<time_unit>,duration=<duration>¶: Service scheduling configuration, which can be configured only for real-time services.

--additional-property <key=value>¶: .Additional service attribute, which facilitates service management Example: –additional-property VAR1=value1 –additional-property VAR2=value2

--model-id <model_id>¶: Model ID

--weight <weight>¶: Traffic weight allocated to a model. This parameter is mandatory only when infer_type is set to real-time. The sum of the weights must be 100.

--cluster-id <cluster_id>¶: ID of a dedicated resource pool. By default, this parameter is left blank, indicating that no dedicated resource pool is used.

--specification <specification>¶: Resource specifications. Select specifications based on service requirements.

--instance-count <instance_count>¶: Number of instances deployed in a model The value must be greater than 0.

--env <key=value>¶: Environment variable key-value pair required for running a model. Example: –env VAR1=value1 –env VAR2=value2

--custom-spec cpu=<cpu>,memory=<memory>,gpu_p4=<gpu_p4>¶: Custom specifications. Set this parameter when you use a dedicated resource pool.

--src-type <src_type>¶: Data source type.

--src-path <src_path>¶: OBS path of the input data of a batch job.

--dest-path <dest_path>¶: OBS path of the output data of a batch job.

--req-uri <dest_path>¶: API URI from the model config.json file for inference. If a ModelArts built-in inference image is used, the value of this parameter is /

--mapping-type {file, csv}¶: Mapping type of the input data.

--mapping-rule <mapping_rule>¶: Mapping between input parameters and CSV data. This parameter is mandatory only when mapping_type is set to csv.

--wait¶: Wait for service deployment.

--timeout <timeout>¶: Timeout for the wait in seconds (default 1200 seconds).

service¶: Service Name or ID

This command is provided by the otcextensions plugin.

Training Job Operations¶

ma training job config create¶

Create a ModelArts Training Job Configuration

openstack ma training job config create
    --worker-server-num <worker_server_num>
    --app-url <app_url>
    --boot-file-url <boot_file_url>
    --spec-id <spec_id>
    --engine-id <engine_id>
    --model-id <model_id>
    [--description <description>]
    [--log-url <log_url>]
    [--data-url <data_url>]
    [--dataset-id <dataset_id>]
    [--dataset-version-id <dataset_version_id>]
    [--train-url <train_url>]
    [--user-image-url <user_image_url>]
    [--user-command <user_command>]
    [--parameter label=<label>,value=<value>]
    <name>

--worker-server-num <worker_server_num>¶: Number of workers in a training job.

--app-url <app_url>¶: Code directory of a training job, for example, /usr/app/.

--boot-file-url <boot_file_url>¶: Boot file of a training job, which needs to be stored in the code directory.

--spec-id <spec_id>¶: ID of the resource specifications selected for a training job.

--engine-id <engine_id>¶: ID of the engine selected for a training job. The default value is 1.

--model-id <model_id>¶: ID of the built-in model of a training job.

--description <description>¶: Description of a training job.

--log-url <log_url>¶: Boot file of a training job, which needs to be stored in the code directory.

--data-url <data_url>¶: OBS URL of the dataset required by a training job.

--dataset-id <dataset_id>¶: Dataset ID of a training job.

--dataset-version-id <dataset_version_id>¶: Dataset version ID of a training job. This parameter must be used together with dataset_id, but cannot be used together with data_url or data_source.

--train-url <train_url>¶: OBS URL of the output file of a training job.

--user-image-url <user_image_url>¶: SWR URL of a custom image used by a training job.

--user-command <user_command>¶: Boot command used to start the container of a custom image ofa training job. The format is bash /home/work/run_train.sh.

--parameter label=<label>,value=<value>¶: Running parameters of a training job. It is a collection of label-value pairs .Example: –parameter label=flower,value=rose –parameter label=color,value=red

name¶: Name of a training job configuration.

This command is provided by the otcextensions plugin.

ma training job config delete¶

Delete ModelArts Training Job(s)

openstack ma training job config delete <name> [<name> ...]

name¶: ModelArts training config name(s) to be deleted.

This command is provided by the otcextensions plugin.

ma training job config list¶

Querying a List of Training Job Configurations.

openstack ma training job config list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--config-type {custom, sample}]
    [--limit <limit>]
    [--offset <offset>]
    [--sort-by {config_name,config_desc,create_time}]
    [--order {asc, desc}]
    [--search-content <search_content>]

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--config-type {custom, sample}¶: Configuration type to be queried. Default value: custom

--limit <limit>¶: Number of jobs displayed on each page. The value range is [1, 1000]. Default value: 10.

--offset <offset>¶: Index of the page to be queried. Default value: 1

--sort-by {config_name,config_desc,create_time}¶: Sorting field. Default value: job_name

--order {asc, desc}¶: Sorting order. Default value: desc

--search-content <search_content>¶: Search content, for example, a training job name.

This command is provided by the otcextensions plugin.

ma training job config show¶

Show details of a modelarts training job configuration.

openstack ma training job config show <name>

name¶: Name of a training job configuration.

This command is provided by the otcextensions plugin.

ma training job create¶

Create a ModelArts Training Job

openstack ma training job create
    --worker-server-num <worker_server_num>
    --app-url <app_url>
    --boot-file-url <boot_file_url>
    --spec-id <spec_id>
    --engine-id <engine_id>
    --model-id <model_id>
    [--description <description>]
    [--workspace-id <workspace_id>]
    [--log-url <log_url>]
    [--data-url <data_url>]
    [--dataset-id <dataset_id>]
    [--dataset-version-id <dataset_version_id>]
    [--train-url <train_url>]
    [--user-image-url <user_image_url>]
    [--user-command <user_command>]
    [--parameter label=<label>,value=<value>]
    [--nfs id=<id>,src_path=<src_path>,dest_path=<dest_path>read_only=<read_only>]
    [--host-path src_path=<src_path>,dest_path=<dest_path>,read_only=<read_only>]
    [--no-create-version]
    <name>

--worker-server-num <worker_server_num>¶: Number of workers in a training job.

--app-url <app_url>¶: Code directory of a training job, for example, /usr/app/.

--boot-file-url <boot_file_url>¶: Boot file of a training job, which needs to be stored in the code directory.

--spec-id <spec_id>¶: ID of the resource specifications selected for a training job.

--engine-id <engine_id>¶: ID of the engine selected for a training job. The default value is 1.

--model-id <model_id>¶: ID of the built-in model of a training job.

--description <description>¶: Description of a training job.

--workspace-id <workspace_id>¶: Workspace where a job resides. Default value: 0

--log-url <log_url>¶: Boot file of a training job, which needs to be stored in the code directory.

--data-url <data_url>¶: OBS URL of the dataset required by a training job.

--dataset-id <dataset_id>¶: Dataset ID of a training job.

--dataset-version-id <dataset_version_id>¶: Dataset version ID of a training job. This parameter must be used together with dataset_id, but cannot be used together with data_url or data_source.

--train-url <train_url>¶: OBS URL of the output file of a training job.

--user-image-url <user_image_url>¶: SWR URL of a custom image used by a training job.

--user-command <user_command>¶: Boot command used to start the container of a custom image ofa training job. The format is bash /home/work/run_train.sh.

--parameter label=<label>,value=<value>¶: Running parameters of a training job. It is a collection of label-value pairs .Example: –parameter label=flower,value=rose –parameter label=color,value=red

--nfs id=<id>,src_path=<src_path>,dest_path=<dest_path>read_only=<read_only>¶: Storage volume of the shared file system type. Only the training jobs running in the resource pool with the shared file system network connected support such storage volume.

--host-path src_path=<src_path>,dest_path=<dest_path>,read_only=<read_only>¶: Storage volume of the host file system type. Only training jobs running in the dedicated resource pool support such storage volume.

--no-create-version¶: A version is not created when a training job is created.

name¶: Training job name.

This command is provided by the otcextensions plugin.

ma training job delete¶

Delete ModelArts Training Job(s)

openstack ma training job delete <JobId> [<JobId> ...]

JobId¶: ID of the ModelArts training job(s) to be deleted.

This command is provided by the otcextensions plugin.

ma training job list¶

Get properties of a vm

openstack ma training job list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--limit <limit>]
    [--offset <offset>]
    [--status {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21}]
    [--sort-by {job_name,job_desc,status,duration,version_count,create_time}]
    [--order {asc, desc}]
    [--search-content <search_content>]
    [--workspace-id <workspace_id>]

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--limit <limit>¶: Number of jobs displayed on each page. The value range is [1, 1000]. Default value: 10.

--offset <offset>¶: Index of the page to be queried. Default value: 1

--status <STATUS>¶: Job status. The options are as follows: 0: UNKNOWN 1: INIT 2: IMAGE_CREATING 3: IMAGE_FAILED 4: SUBMIT_TRYING 5: SUBMIT_FAILED 6: DELETE_FAILED 7: WAITING 8: RUNNING 9: KILLING 10: COMPLETED 11: FAILED 12: KILLED 13: CANCELED 14: LOST 15: SCALING 16: SUBMIT_MODEL_FAILED 17: DEPLOY_SERVICE_FAILED 18: CHECK_INIT 19: CHECK_RUNNING 20: RUNNING_COMPLETED 21: CHECK_FAILED

--sort-by {job_name,job_desc,status,duration,version_count,create_time}¶: Sorting field. Default value: job_name

--order {asc, desc}¶: Sorting order. Default value: desc

--search-content <search_content>¶: Search content, for example, a training job name.

--workspace-id <workspace_id>¶: Workspace ID.

This command is provided by the otcextensions plugin.

ma training job update¶

Modify details of a modelarts training job.

openstack ma training job update --description <description> <jobId>

--description <description>¶: Description of a training job.

jobId¶: Enter training job id.

This command is provided by the otcextensions plugin.

ma training job version create¶

Create a ModelArts Training Job Version

openstack ma training job version create
    --worker-server-num <worker_server_num>
    --app-url <app_url>
    --boot-file-url <boot_file_url>
    --spec-id <spec_id>
    --engine-id <engine_id>
    --model-id <model_id>
    --pre-version-id <pre_version_id>
    [--description <description>]
    [--log-url <log_url>]
    [--data-url <data_url>]
    [--dataset-id <dataset_id>]
    [--dataset-version-id <dataset_version_id>]
    [--train-url <train_url>]
    [--user-image-url <user_image_url>]
    [--user-command <user_command>]
    [--parameter label=<label>,value=<value>]
    <jobId>

--worker-server-num <worker_server_num>¶: Number of workers in a training job.

--app-url <app_url>¶: Code directory of a training job, for example, /usr/app/.

--boot-file-url <boot_file_url>¶: Boot file of a training job, which needs to be stored in the code directory.

--spec-id <spec_id>¶: ID of the resource specifications selected for a training job.

--engine-id <engine_id>¶: ID of the engine selected for a training job. The default value is 1.

--model-id <model_id>¶: ID of the built-in model of a training job.

--pre-version-id <pre_version_id>¶: ID of the previous version of a training job.

--description <description>¶: Description of a training job.

--log-url <log_url>¶: Boot file of a training job, which needs to be stored in the code directory.

--data-url <data_url>¶: OBS URL of the dataset required by a training job.

--dataset-id <dataset_id>¶: Dataset ID of a training job.

--dataset-version-id <dataset_version_id>¶: Dataset version ID of a training job. This parameter must be used together with dataset_id, but cannot be used together with data_url or data_source.

--train-url <train_url>¶: OBS URL of the output file of a training job.

--user-image-url <user_image_url>¶: SWR URL of a custom image used by a training job.

--user-command <user_command>¶: Boot command used to start the container of a custom image ofa training job. The format is bash /home/work/run_train.sh.

--parameter label=<label>,value=<value>¶: Running parameters of a training job. It is a collection of label-value pairs .Example: –parameter label=flower,value=rose –parameter label=color,value=red

jobId¶: Training job ID.

This command is provided by the otcextensions plugin.

ma training job version delete¶

Delete ModelArts training job version.

openstack ma training job version delete <jobId> <versionId>

jobId¶: ID of the training job.

versionId¶: Version id of the training job.

This command is provided by the otcextensions plugin.

ma training job version list¶

Get the list of training job versions

openstack ma training job version list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--limit <limit>]
    [--offset <offset>]
    <jobId>

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--limit <limit>¶: Number of jobs displayed on each page. The value range is [1, 1000]. Default value: 10.

--offset <offset>¶: Index of the page to be queried. Default value: 1

jobId¶: ID of the training job.

This command is provided by the otcextensions plugin.

ma training job version logfile¶

Obtain the name of a training job log file

openstack ma training job version logfile
    --job_id <job_id>
    --version_id <version_id>

--job_id <job_id>¶: ID of a training job

--version_id <version_id>¶: Version ID of a training job

This command is provided by the otcextensions plugin.

ma training job version logs list¶

Query detailed information about training job logs by row

openstack ma training job version logs list
    --job_id <job_id>
    --version_id <version_id>
    [--base_line <base_line>]
    [--lines <lines>]
    --log_file <log_file>
    [--order <order>]

--job_id <job_id>¶: ID of a training job

--version_id <version_id>¶: Version ID of a training job

--base_line <base_line>¶: Base line of the log, which is obtained from an API response

--lines <lines>¶: Length of the obtained log

--log_file <log_file>¶: Name of the log file to be viewed

--order <order>¶: Log query directiondesc: Query next records

This command is provided by the otcextensions plugin.

ma training job version show¶

Show details of a modelarts Training Job Version

openstack ma training job version show <jobId> <versionId>

jobId¶: ID of a training job

versionId¶: Version ID of a training job.

This command is provided by the otcextensions plugin.

ma training job version stop¶

Stop Training Job Version.

openstack ma training job version stop <jobId> <versionId>

jobId¶: ID of a training job

versionId¶: ID of a training job version

This command is provided by the otcextensions plugin.

Visualization Job Operations¶

ma visualization job create¶

Create a Visualization Job model

openstack ma visualization job create
    [--description <description>]
    --train-url <train_url>
    [--job-type <job_type>]
    [--flavor <flavor>]
    [--auto-stop-duration <auto_stop_duration>]
    <name>

--description <description>¶: Description of a visualization job.

--train-url <train_url>¶: OBS path

--job-type <job_type>¶: Type of a visualization job

--flavor <flavor>¶: Specifications when a visualization job is created.

--auto-stop-duration <auto_stop_duration>¶: Auto stop duration. The value ranges from 0 to 24.

name¶: Name of a visualization job.

This command is provided by the otcextensions plugin.

ma visualization job delete¶

Delete ModelArts visualization Job(s)

openstack ma visualization job delete <JobId> [<JobId> ...]

JobId¶: ID of the ModelArts visualization job(s) to be deleted.

This command is provided by the otcextensions plugin.

ma visualization job list¶

Query the visualization jobs that meet the search criteria.

openstack ma visualization job list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--limit <limit>]
    [--offset <offset>]
    [--status {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21}]
    [--sort-by {job_id,job_name,job_desc,status,create_time}]
    [--order {asc, desc}]
    [--search-content <search_content>]
    [--workspace-id <workspace_id>]

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--limit <limit>¶: Number of jobs displayed on each page. The value range is [1, 1000]. Default value: 10.

--offset <offset>¶: Index of the page to be queried. Default value: 1

--status <STATUS>¶: Job status. The options are as follows: 0: UNKNOWN 1: INIT 2: IMAGE_CREATING 3: IMAGE_FAILED 4: SUBMIT_TRYING 5: SUBMIT_FAILED 6: DELETE_FAILED 7: WAITING 8: RUNNING 9: KILLING 10: COMPLETED 11: FAILED 12: KILLED 13: CANCELED 14: LOST 15: SCALING 16: SUBMIT_MODEL_FAILED 17: DEPLOY_SERVICE_FAILED 18: CHECK_INIT 19: CHECK_RUNNING 20: RUNNING_COMPLETED 21: CHECK_FAILED

--sort-by {job_id,job_name,job_desc,status,create_time}¶: Sorting field. Default value: job_name

--order {asc, desc}¶: Sorting order. Default value: desc

--search-content <search_content>¶: Search content, for example, a training job name.

--workspace-id <workspace_id>¶: Workspace ID.

This command is provided by the otcextensions plugin.

ma visualization job restart¶

Restart Visualization Job

openstack ma visualization job restart <jobId>

jobId¶: ID of a visualization job

This command is provided by the otcextensions plugin.

ma visualization job show¶

Show details of a visualization job

openstack ma visualization job show <jobId>

jobId¶: ID of a visualization job

This command is provided by the otcextensions plugin.

ma visualization job stop¶

Stop Visualization Job

openstack ma visualization job stop <jobId>

jobId¶: ID of a visualization job

This command is provided by the otcextensions plugin.

ma visualization job update¶

Modify Visualization Job Description

openstack ma visualization job update
    --description <description>
    <jobId>

--description <description>¶: Description of a visualization job

jobId¶: ID of a visualization job

This command is provided by the otcextensions plugin.

Dataset Operations¶

ma dataset create¶

Create a modelarts dataset

openstack ma dataset create
    --dataset-type <dataset_type>
    --data-source data_type
    =<data_type>,data_path=<data_path>
    [--description <description>]
    [--import-annotations]
    [--import-data]
    [--label-format label_type=<label_type>,text_label_separator=<text_label_separator>,text_sample_separator=<text_sample_separator>]
    [--managed]
    --work-path <work_path>
    [--work-path-type <work_path_type>]
    [--workspace-id <workspace_id>]
    <name>

--dataset-type <dataset_type>¶: Dataset type. Possible values: 0: image classification 1: object detection 100: text classification 101: named entity recognition 102: text triplet 200: sound classification 201: speech content 202: speech paragraph labeling 400: table dataset 600: video labeling 900: custom format

--data-source data_type=<data_type>,data_path=<data_path>¶: Input dataset path and type. The following keys are required: data_type=<data_type>: Input data source type data_path=<data_path>: PData source path.

--description <description>¶: Dataset description.

--import-annotations¶: Whether to automatically import the labeling information in the input directory.

--import-data¶: Whether to import data. This parameter is used only for table datasets.

--label-format label_type=<label_type>,text_label_separator=<text_label_separator>,text_sample_separator=<text_sample_separator>¶: Label format information. This parameter is used only for text datasets.label_type=<label_type>: Label type of text classification. text_label_separator=<text_label_separator>: Separator between labels. By default, a comma (,) is used as the separator. text_sample_separator=<text_sample_separator>: Separator between the text and label. By default, the Tab key is used as the separator.

--managed¶: Whether to host a dataset.

--work-path <work_path>¶: Output dataset path, which is used to store output files such as label files. The format is /Bucket name/File path, for example, /obs-bucket/flower/rose/.

--work-path-type <work_path_type>¶: Type of the dataset output path. The options are as follows: 0: OBS bucket (default value)

--workspace-id <workspace_id>¶: Workspace ID.

name¶: Dataset name.

This command is provided by the otcextensions plugin.

ma dataset delete¶

Delete Modelarts Dataset(s).

openstack ma dataset delete <dataset> [<dataset> ...]

dataset¶: Name or Id of the dataset(s) to be deleted.

This command is provided by the otcextensions plugin.

ma dataset export task list¶

Get properties of a vm

openstack ma dataset export task list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--limit <limit>]
    [--offset <offset>]
    [--export-type {0,1,2,4}]
    <dataset>

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--limit <limit>¶: Maximum number of records returned on each page. The default value is 10.

--offset <offset>¶: Start page of the paging list. The default value is 0.

--export-type <EXPORT_TYPE>¶: Export type. The options are as follows: 0: labeled 1: unlabeled 2: all 4: conditional search

dataset¶: Dataset Id or name.

This command is provided by the otcextensions plugin.

ma dataset export task show¶

Show details of a MA dataset export task

openstack ma dataset export task show <dataset> <taskId>

dataset¶: Dataset Id or name.

taskId¶: Enter task Id.

This command is provided by the otcextensions plugin.

ma dataset import task list¶

Get properties of a vm

openstack ma dataset import task list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--limit <limit>]
    [--offset <offset>]
    <dataset>

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--limit <limit>¶: Maximum number of records returned on each page. The default value is 10.

--offset <offset>¶: Start page of the paging list. The default value is 0.

dataset¶: Dataset Id or Name.

This command is provided by the otcextensions plugin.

ma dataset import task show¶

Show details of a MA dataset import task

openstack ma dataset import task show <dataset> <taskId>

dataset¶: Enter dataset name or Id.

taskId¶: Enter task id.

This command is provided by the otcextensions plugin.

ma dataset label list¶

This API is used to query all samples of a dataset.

openstack ma dataset label list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--version-id <version_id>]
    <dataset>

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--version-id <version_id>¶: Dataset version id.

dataset¶: Dataset Id or name.

This command is provided by the otcextensions plugin.

ma dataset list¶

Get List of Modelarts Datasets.

openstack ma dataset list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--check-running-task]
    [--contain-versions]
    [--dataset-type <dataset_type>]
    [--file-preview]
    [--limit <limit>]
    [--offset <offset>]
    [--order {asc, desc}]
    [--running-task-type <running_task_type>]
    [--search-content <search_content>]
    [--sort-by {create_time, dataset_name}]
    [--support-export]
    [--train-evaluate-ratio <train_evaluate_ratio>]
    [--version-format <version_format>]
    [--with-labels]
    [--workspace-id <workspace_id>]

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--check-running-task¶: Whether to detect tasks (including initialization tasks) that are running in a dataset.

--contain-versions¶: Whether the dataset contains a version.

--dataset-type <dataset_type>¶: Dataset type. Possible values: 0: image classification 1: object detection 100: text classification 101: named entity recognition 102: text triplet 200: sound classification 201: speech content 202: speech paragraph labeling 400: table dataset 600: video labeling 900: custom format

--file-preview¶: Whether a dataset supports preview when it is queried.

--limit <limit>¶: Maximum number of records returned on each page. The default value is 10.

--offset <offset>¶: Start page of the paging list. The default value is 0.

--order {asc, desc}¶: Sorting order. Default value: desc

--running-task-type <running_task_type>¶: Type of the running tasks (including initialization tasks) to be detected. The options are as follows: 0: auto labeling 1: pre-labeling 2: export 3: version switch 4: manifest file export 5: manifest file import 6: version publishing 7: auto grouping 10: one-click model deployment (default value)

--search-content <search_content>¶: Fuzzy search keyword.

--sort-by {create_time, dataset_name}¶: Sorting field. Default value: publish_at

--support-export¶: Whether to filter datasets that can be exported only (including datasets of image classification, object detection, and custom format).

--train-evaluate-ratio <train_evaluate_ratio>¶: Version split ratio for dataset filtering.

--version-format <version_format>¶: Dataset version format for dataset filtering. This parameter is used to filter datasets that meet the filter criteria. The options are as follows: 0: default format 1: CarbonData (supported only by table datasets) 2: CSV

--with-labels¶: Whether to return dataset labels.

--workspace-id <workspace_id>¶: Workspace ID.

This command is provided by the otcextensions plugin.

ma dataset sample add¶

Add samples to a dataset.

openstack ma dataset sample add
    [--file-path <file_path>]
    [--directory-path <directory_path>]
    [--encoding {UTF-8, GBK, GB2312}]
    [--metadata <key=value>]
    [--sample-type {0,1,2,4,6,9}]
    [--data-source-path <data_source_path>]
    [--data-source-type {0,1,2,3,4,5,6}]
    [--data-source-info <key1=value1,key2=value2,..>]
    [--data-with-column-header]
    [--schema-map src_name=<src_name>,dest_name=<dest_name>]
    [--label key1=<value1>,key2=<value2>,..]
    [--to-be-confirmed]
    <dataset>

--file-path <file_path>¶: Sample file path to upload.

--directory-path <directory_path>¶: Samples Directory path.

--encoding {UTF-8, GBK, GB2312}¶: Encoding type of sample files, which is used to upload .txt or .csv files. The default value is UTF-8.

--metadata <key=value>¶: Key-value pair of the sample metadata attribute. Repeat the option to set multiple metadata attributes. Example: –metadata @modelarts:hard=0 –metadata @modelarts:hard_coefficient=0.22 –metadata @modelarts:hard_reasons=[1,2] –metadata @modelarts:size=[100,200,3]

--sample-type <SAMPLE_TYPE>¶: Sample type. The options are as follows: 0: Image 1: Text 2: Speech 4: Table 6: Video 9: Custom_Format

--data-source-path <data_source_path>¶: Data source path.

--data-source-type <DATA_SOURCE_TYPE>¶: Data source type. The options are as follows: 0: OBS bucket (default value) 1: GaussDB(DWS) 2: DLI 3: RDS 4: MRS 5: AI Gallery 6: Inference service

--data-source-info <key1=value1,key2=value2,..>¶: Information required for importing a table data source. Supported keys: cluster_id: ID of a MRS cluster cluster_mode: Running mode of a MRS cluster. Options are as follows: * 0: normal cluster * 1: security cluster cluster_name: Name of a MRS cluster database_name: Name of the database to which the table dataset is imported. input: HDFS path of a table dataset ip: IP address of your GaussDB(DWS) cluster port: Port number of your GaussDB(DWS) cluster queue_name: DLI queue name of a table dataset subnet_id: Subnet ID of a MRS cluster table_name: Name of the table to which a table dataset is imported user_name: Username which is mandatary for GaussDB(DWS) user_password: User password which is mandatary for GaussDB(DWS) vpc_id: ID of the vpc of a MRS cluster Example: –data-source-info cluster_id=1234,cluster_mode=1

--data-with-column-header¶: Whether the first row in the data file is a column name.

--schema-map src_name=<src_name>,dest_name=<dest_name>¶: Schema mapping information corresponding to the table data. src_name: Name of the source column. dest_name: Name of the destination column. Example: –schema-map src_name=colnameA,dest_name=colnameB –schema-map src_name=colnameX,dest_name=colnameY

--label key1=<value1>,key2=<value2>,..¶: Sample label list. Supported keys: annotated_by: Video labeling method, which is used to distinguish whether a video is labeled manually or automatically.The options are as follows: * human: manual labeling * auto: automatic labeling id: Label ID name: Label name score: Confidence type: Label type. The options are as follows: * 0: image classification * 1: object detection * 100: text classification * 101: named entity recognition * 102: text triplet relationship * 103: text triplet entity* 200: speech classification* 201: speech content* 202: speech paragraph labeling* 600: video classification Example: –label cluster_id=1234,cluster_mode=1

--to-be-confirmed¶: Whether to import labels to the to-be-confirmed dataset. Currently, to-be-confirmed datasets only support categories of image classification and object detection.

dataset¶: Dataset Id or name.

This command is provided by the otcextensions plugin.

ma dataset sample delete¶

This API is used to delete the samples of a dataset in batches.

openstack ma dataset sample delete
    [--delete-source]
    <dataset>
    <sampleId>
    [<sampleId> ...]

--delete-source¶: Whether to delete the source file. (default: false.)

dataset¶: Dataset Id or name.

sampleId¶: ID of Dataset sample(s) to be deleted.

This command is provided by the otcextensions plugin.

ma dataset sample list¶

Get details about a Sample.

openstack ma dataset sample list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--email <email>]
    [--high-score <high_score>]
    [--label-name <label_name>]
    [--label-type {0,1,100,101,102,200,201,202,400,600,900}]
    [--limit <limit>]
    [--locale <locale>]
    [--low-score <low_score>]
    [--offset <offset>]
    [--order {asc, desc}]
    [--preview]
    [--process-parameter <process_parameter>]
    [--sample-state {all,none,uncheck,accepted,rejected,unreviewed,reviewed,workforce_sampled,workforce_sampled_uncheck,workforce_sampled_checked,workforce_sampled_accepted,workforce_sampled_rejected,auto_annotation}]
    [--sample-type {0,1,2,4,6,9}]
    [--search-conditions <search_conditions>]
    [--version-id <version_id>]
    <dataset>

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--email <email>¶: Email address of a labeling team member.

--high-score <high_score>¶: Upper confidence limit. The default value is 1.

--label-name <label_name>¶: Label name.

--label-type <LABEL_TYPE>¶: Labeling type. The options are as follows:0: image classification 1: object detection 100: text classification 101: named entity recognition 102: text triplet 200: sound classification 201: speech content 202: speech paragraph labeling 400: table dataset 600: video labeling 900: custom format

--limit <limit>¶: Maximum number of records returned on each page. The value ranges from 1 to 100. The default value is 10.

--locale <locale>¶: Language. The options are as follows: en-us: English (default value).

--low-score <low_score>¶: Lower confidence limit. The default value is 0.

--offset <offset>¶: Start page of the paging list. The default value is 0.

--order {asc, desc}¶: Sorting order. Default value: desc

--preview¶: Whether to support preview.

--process-parameter <process_parameter>¶: Image resizing setting, which is the same as the OBS resizing setting.

--sample-state {all,none,uncheck,accepted,rejected,unreviewed,reviewed,workforce_sampled,workforce_sampled_uncheck,workforce_sampled_checked,workforce_sampled_accepted,workforce_sampled_rejected,auto_annotation}¶: Sample status.

--sample-type <SAMPLE_TYPE>¶: Sample type. The options are as follows:0: Image 1: Text 2: Speech 4: Table 6: Video 9: Custom_Format

--search-conditions <search_conditions>¶: Multi-dimensional search condition after URL encoding. The relationship between multiple search conditions is AND.

--version-id <version_id>¶: Dataset version ID.

dataset¶: Dataset Id or name.

This command is provided by the otcextensions plugin.

ma dataset sample show¶

Show instance details

openstack ma dataset sample show <dataset> <sampleId>

dataset¶: Dataset Id or name.

sampleId¶: Dataset Sample ID.

This command is provided by the otcextensions plugin.

ma dataset show¶

Show details of a Modelarts Dataset.

openstack ma dataset show <dataset>

dataset¶: Dataset Id or name.

This command is provided by the otcextensions plugin.

ma dataset statistics¶

Show statistics of a Modelarts dataset.

openstack ma dataset statistics <dataset>

dataset¶: Dataset Id or name.

This command is provided by the otcextensions plugin.

ma dataset sync¶

Creating a Dataset Synchronization Task

openstack ma dataset sync <dataset>

dataset¶: Dataset Id or name.

This command is provided by the otcextensions plugin.

ma dataset sync status¶

Show details of a Modelarts Dataset.

openstack ma dataset sync status <dataset>

dataset¶: Dataset Id or name.

This command is provided by the otcextensions plugin.

ma dataset update¶

Modify details of a modelarts Dataset.

openstack ma dataset update [--description <description>] <dataset>

--description <description>¶: Enter description.

dataset¶: Name or Id of the dataset.

This command is provided by the otcextensions plugin.

ma dataset version create¶

Create a ModelArts dataset version

openstack ma dataset version create
    [--name <name>]
    [--description <description>]
    [--version-format <version_format>]
    [--remove-sample-usage <remove_sample_usage>]
    [--export-images <export_images>]
    [--train-evaluate-sample-ratio <train_evaluate_sample_ratio>]
    [--clear_hard_property <clear_hard_property>]
    <dataset>

--name <name>¶: version_name. The value is a string of 1 to 32 characters consisting of only digits, letters, underscores (_), and hyphens (-). Example value: dataset

--description <description>¶: Dataset description. The value is a string of 0 to 256 characters. By default, this parameter is left blank.

--version-format <version_format>¶: Format of the exported version file, which is case insensitive.

--remove-sample-usage <remove_sample_usage>¶: Whether to clear the usage information of dataset samples. The default value is true.

--export-images <export_images>¶: Whether to export images to the version output directory during publishing. The default value is false.

--train-evaluate-sample-ratio <train_evaluate_sample_ratio>¶: Ratio that splits the labeled data into training and validation sets during publishing. The value must be a decimal between 0 and 1.

--clear_hard_property <clear_hard_property>¶: Whether to clear hard example properties. The default value is true.

dataset¶: Dataset Id or name.

This command is provided by the otcextensions plugin.

ma dataset version delete¶

Delete ModelArts Dataset Version.

openstack ma dataset version delete <dataset> <versionId>

dataset¶: Dataset Id or name.

versionId¶: Version Id of the dataset.

This command is provided by the otcextensions plugin.

ma dataset version list¶

This API is used to query the Dataset Version list.

openstack ma dataset version list
    [--sort-column SORT_COLUMN]
    [--sort-ascending | --sort-descending]
    [--status <status>]
    [--train-evaluate-ratio <train_evaluate_ratio>]
    [--version-format <version_format>]
    <dataset>

--sort-column SORT_COLUMN¶: specify the column(s) to sort the data (columns specified first have a priority, non-existing columns are ignored), can be repeated

--sort-ascending¶: sort the column(s) in ascending order

--sort-descending¶: sort the column(s) in descending order

--status <status>¶: Status of a dataset version. The options are as follows: 0: creating 1: running 2: deleting 3: deleted 4: error

--train-evaluate-ratio <train_evaluate_ratio>¶: Version split ratio for version filtering.

--version-format <version_format>¶: Format of a dataset version. The options are as follows: 0: default format 1: CarbonData (supported only by table datasets) 2: CSV

dataset¶: Dataset Id or name.

This command is provided by the otcextensions plugin.

ma dataset version show¶

Show details of a MA dataset version

openstack ma dataset version show <dataset> <versionId>

dataset¶: Dataset Id or name.

versionId¶: Dataset version id.

This command is provided by the otcextensions plugin.

last updated: UTC - commit: