Querying the Team Labeling Task List of a Dataset

Function

This API is used to query the team labeling task list of a dataset.

Debugging

You can debug this API through automatic authentication in or use the SDK sample code generated by API Explorer.

URI

GET /v2/{project_id}/datasets/{dataset_id}/workforce-tasks

Table 1 Path Parameters

Parameter

Mandatory

Type

Description

dataset_id

Yes

String

Dataset ID.

project_id

Yes

String

Project ID. For details about how to obtain a project ID, see Obtaining a Project ID and Name.

Table 2 Query Parameters

Parameter

Mandatory

Type

Description

offset

No

Integer

Start page for pagination display. The default value is 0.

limit

No

Integer

Maximum number of records returned on each page. The value ranges from 1 to 1000. The default value is 1000.

Request Parameters

None

Response Parameters

Status code: 200

Table 3 Response body parameters

Parameter

Type

Description

count

Integer

Total number of team labeling tasks.

tasks

Array of WorkforceTask objects

Team labeling task list queried by page.

Table 4 WorkforceTask

Parameter

Type

Description

auto_sync_dataset

Boolean

Whether to automatically synchronize the result of a team labeling task to the dataset. Options:

  • true: Automatically synchronize the result of a team labeling task to the dataset.

  • false: Do not automatically synchronize the result of a team labeling task to the dataset.

check_rate

Double

Acceptance ratio of a team labeling task.

checking_task_desc

WorkforceSamplingTaskDesc object

Details about the current acceptance task of a team labeling task.

create_time

Long

Time when a labeling task is created.

dataset_id

String

Dataset ID.

description

String

Description of a labeling task.

label_stats

Array of LabelStats objects

Label statistics of a labeling task.

pass_rate

Double

Acceptance pass rate of a team labeling task.

repetition

Integer

Number of persons who label each sample in a team labeling task. The minimum value is 1.

sample_search_conditions

Array of SearchCondition objects

Sample search criteria when creating a task.

sample_stats

SampleStats object

Sample statistics of a labeling task.

score

Double

Average acceptance score of a team labeling task.

status

Integer

Status of a team labeling task. Options:

  • 0: Starting. The owner or manager is starting the task, and files to be labeled are still being allocated. The owner, manager, annotator, and reviewer can view the task list. Since the task allocation has not been completed, no other task can be started.

  • 1: Running. The annotator or reviewer performs labeling or reviews the task, and the owner performs acceptance. For example, new files need to be allocated again if auto labeling files are added or synchronized, or unlabeled files are imported.

  • 2: Accepting. The owner initiates an acceptance task, but the acceptance has not been completed. In this state, a new acceptance task can be started only after the current one is completed.

  • 3: Accepted. The team labeling task has been completed.

  • **4: Rejected. The manager starts the task again. In this state, samples need to be labeled and the task must be reviewed again.

  • 5: Synchronizing acceptance result. This status is added when an acceptance task changes to be asynchronous. In this state, no new acceptance task can be initiated and the current acceptance task cannot be continued. In the task name area, the system displays a message indicating that the acceptance result is being synchronized.

  • 6: Created. The task has been created by the owner but has not been started. Only the owner and manager can view the task list.

  • 7: Acceptance sampling. This status is added when an acceptance task changes to be asynchronous. In this state, no new acceptance task can be initiated and the current acceptance task cannot be continued. In the task name area, the system displays a message indicating that acceptance sampling is in progress.

synchronize_auto_labeling_data

Boolean

Whether to synchronize the auto labeling result of a team labeling task. Options:

  • true: Synchronize the results to be confirmed to team members after auto labeling is complete.

  • false: Do not synchronize the auto labeling results. (Default value)

synchronize_data

Boolean

Whether to synchronize the added data of a team labeling task. Options:

  • true: Synchronize uploaded files, data sources, and imported unlabeled files to team members.

  • false: Do not synchronize the added data. (Default value)

task_id

String

ID of a labeling task.

task_name

String

Name of a labeling task.

update_time

Long

Time when a labeling task is updated.

version_id

String

Version ID of the dataset associated with a labeling task.

workforce_stats

WorkforceStats object

Statistics on team labeling task members.

workforces_config

WorkforcesConfig object

Team labeling task information: Tasks can be assigned by the team administrator or a specified team.

Table 5 WorkforceSamplingTaskDesc

Parameter

Type

Description

action

Integer

Acceptance operations. Options:

  • 0: Pass all samples.

  • 1: Reject all samples.

  • 4: Pass single-accepted samples and unaccepted samples.

  • 5: Reject single-rejected samples and unaccepted samples.

checking_stats

CheckTaskStats object

Real-time report of acceptance tasks.

checking_task_id

String

ID of the current acceptance task.

overwrite_last_result

Boolean

Whether to use the acceptance result to overwrite the labeled result if a sample has been labeled during acceptance. Options:

  • true: Overwrite the labeled result.

  • false: Do not overwrite the labeled result. (Default value)

total_stats

CheckTaskStats object

Overall report of historical acceptance tasks.

Table 6 CheckTaskStats

Parameter

Type

Description

accepted_sample_count

Integer

Accepted samples.

checked_sample_count

Integer

Checked samples.

pass_rate

Double

Pass rate of samples.

rejected_sample_count

Integer

Rejected samples.

sampled_sample_count

Integer

Number of sampled samples.

sampling_num

Integer

Samples of an acceptance task.

sampling_rate

Double

Sampling rate of an acceptance task.

score

String

Acceptance score.

task_id

String

ID of an acceptance task.

total_sample_count

Integer

Total samples.

total_score

Long

Total acceptance score.

unchecked_sample_count

Integer

Unchecked samples.

Table 7 LabelStats

Parameter

Type

Description

attributes

Array of LabelAttribute objects

Multi-dimensional attribute of a label. For example, if the label is music, attributes such as style and artist may be included.

count

Integer

Number of labels.

name

String

Label name.

property

LabelProperty object

Basic attribute key-value pair of a label, such as color and shortcut keys.

sample_count

Integer

Number of samples containing the label.

type

Integer

Label type. Options:

  • 0: image classification

  • 1: object detection

  • 3: image segmentation

  • 100: text classification

  • 101: named entity recognition

  • 102: text triplet relationship

  • 103: text triplet entity

  • 200: sound classification

  • 201: speech content

  • 202: speech paragraph labeling

  • 600: video labeling

Table 8 LabelAttribute

Parameter

Type

Description

default_value

String

Default value of a label attribute.

id

String

Label attribute ID.

name

String

Label attribute name.

type

String

Label attribute type. Options:

  • text: text

  • select: single-choice drop-down list

values

Array of LabelAttributeValue objects

List of label attribute values.

Table 9 LabelAttributeValue

Parameter

Type

Description

id

String

Label attribute value ID.

value

String

Label attribute value.

Table 10 LabelProperty

Parameter

Type

Description

@modelarts:color

String

Default attribute: Label color, which is a hexadecimal code of the color. By default, this parameter is left blank. Example: #FFFFF0.

@modelarts:default_shape

String

Default attribute: Default shape of an object detection label (dedicated attribute). By default, this parameter is left blank. Options:

  • bndbox: rectangle

  • polygon: polygon

  • circle: circle

  • line: straight line

  • dashed: dotted line

  • point: point

  • polyline: polyline

@modelarts:from_type

String

Default attribute: Type of the head entity in the triplet relationship label. This attribute must be specified when a relationship label is created. This parameter is used only for the text triplet dataset.

@modelarts:rename_to

String

Default attribute: The new name of the label.

@modelarts:shortcut

String

Default attribute: Label shortcut key. By default, this parameter is left blank. For example: D.

@modelarts:to_type

String

Default attribute: Type of the tail entity in the triplet relationship label. This attribute must be specified when a relationship label is created. This parameter is used only for the text triplet dataset.

Table 11 SearchCondition

Parameter

Type

Description

coefficient

String

Filter by coefficient of difficulty.

frame_in_video

Integer

A frame in the video.

hard

String

Whether a sample is a hard sample. Options:

  • 0: non-hard sample

  • 1: hard sample

import_origin

String

Filter by data source.

kvp

String

CT dosage, filtered by dosage.

label_list

SearchLabels object

Label search criteria.

labeler

String

Labeler.

metadata

SearchProp object

Search by sample attribute.

parent_sample_id

String

Parent sample ID.

sample_dir

String

Directory where data samples are stored (the directory must end with a slash (/)). Only samples in the specified directory are searched for. Recursive search of directories is not supported.

sample_name

String

Search by sample name, including the file name extension.

sample_time

String

When a sample is added to the dataset, an index is created based on the last modification time (accurate to day) of the sample on OBS. You can search for the sample based on the time. Options:

  • month: Search for samples added from 30 days ago to the current day.

  • day: Search for samples added from yesterday (one day ago) to the current day.

  • yyyyMMdd-yyyyMMdd: Search for samples added in a specified period (at most 30 days), in the format of Start date-End date. For example, 20190901-2019091501 indicates that samples generated from September 1 to September 15, 2019 are searched.

score

String

Search by confidence.

slice_thickness

String

DICOM layer thickness. Samples are filtered by layer thickness.

study_date

String

DICOM scanning time.

time_in_video

String

A time point in the video.

Table 12 SearchLabels

Parameter

Type

Description

labels

Array of SearchLabel objects

List of label search criteria.

op

String

If you want to search for multiple labels, op must be specified. If you search for only one label, op can be left blank. Options:

  • OR: OR operation

  • AND: AND operation

Table 13 SearchLabel

Parameter

Type

Description

name

String

Label name.

op

String

Operation type between multiple attributes. Options:

  • OR: OR operation

  • AND: AND operation

property

Map<String,Array<String>>

Label attribute, which is in the Object format and stores any key-value pairs. key indicates the attribute name, and value indicates the value list. If value is null, the search is not performed by value. Otherwise, the search value can be any value in the list.

type

Integer

Label type. Options:

  • 0: image classification

  • 1: object detection

  • 3: image segmentation

  • 100: text classification

  • 101: named entity recognition

  • 102: text triplet relationship

  • 103: text triplet entity

  • 200: sound classification

  • 201: speech content

  • 202: speech paragraph labeling

  • 600: video labeling

Table 14 SearchProp

Parameter

Type

Description

op

String

Relationship between attribute values. Options:

  • AND: AND relationship

  • OR: OR relationship

props

Map<String,Array<String>>

Search criteria of an attribute. Multiple search criteria can be set.

Table 15 SampleStats

Parameter

Type

Description

accepted_sample_count

Integer

Number of samples accepted by the owner.

auto_annotation_sample_count

Integer

Number of samples to be confirmed after intelligent labeling.

deleted_sample_count

Integer

Number of deleted samples.

rejected_sample_count

Integer

Number of samples that failed to pass the owner acceptance.

sampled_sample_count

Integer

Number of samples that are to be accepted by the owner and sampled.

total_sample_count

Integer

Total number of samples.

unannotated_sample_count

Integer

Number of unlabeled samples.

uncheck_sample_count

Integer

Number of samples that have been approved by the reviewer and are to be accepted by the owner.

unreviewed_sample_count

Integer

Number of samples that have been labeled by the labeler but have not been reviewed by the reviewer.

Table 16 WorkforceStats

Parameter

Type

Description

labeler_count

Integer

Number of labeling persons.

reviewer_count

Integer

Number of reviewers.

workforce_count

Integer

Number of teams.

Table 17 WorkforcesConfig

Parameter

Type

Description

agency

String

Administrator

workforces

Array of WorkforceConfig objects

List of teams that execute labeling tasks.

Table 18 WorkforceConfig

Parameter

Type

Description

workers

Array of Worker objects

List of labeling team members.

workforce_id

String

ID of a labeling team.

workforce_name

String

Name of a labeling team. The value contains 0 to 1024 characters and does not support the following special characters: !<>=&"'

Table 19 Worker

Parameter

Type

Description

create_time

Long

Creation time.

description

String

Labeling team member description. The value contains 0 to 256 characters and does not support the following special characters: ^!<>=&"'

email

String

Email address of a labeling team member.

role

Integer

Role. Options:

  • 0: labeling personnel

  • 1: reviewer

  • 2: team administrator

  • 3: dataset owner

status

Integer

Current login status of a labeling team member. Options:

  • 0: The invitation email has not been sent.

  • 1: The invitation email has been sent but the user has not logged in.

  • 2: The user has logged in.

  • 3: The labeling team member has been deleted.

update_time

Long

Update time.

worker_id

String

ID of a labeling team member.

workforce_id

String

ID of a labeling team.

Example Requests

Querying the Team Labeling Task List of a Dataset

GET https://{endpoint}/v2/{project_id}/datasets/{dataset_id}/workforce-tasks

Example Responses

Status code: 200

OK

{
  "count" : 2,
  "tasks" : [ {
    "dataset_id" : "WxCREuCkBSAlQr9xrde",
    "task_id" : "tY330MHxV9dqIPVaTRM",
    "task_name" : "task-cd60",
    "status" : 1,
    "create_time" : 1606224714358,
    "update_time" : 1606224714358,
    "repetition" : 1,
    "workforces_config" : {
      "workforces" : [ {
        "workforce_id" : "0PfqwXA8M59pppYBx4k",
        "workforce_name" : "team-123",
        "workers" : [ {
          "email" : "xxx@xxx.com",
          "worker_id" : "6db04ae0afb54d7274a2982255516c29",
          "role" : 2
        }, {
          "email" : "xxx@xxx.com",
          "worker_id" : "8c15ad080d3eabad14037b4eb00d6a6f",
          "role" : 0
        } ]
      } ]
    },
    "synchronize_data" : true,
    "synchronize_auto_labeling_data" : true,
    "workforce_stats" : {
      "workforce_count" : 1,
      "labeler_count" : 1,
      "reviewer_count" : 0
    },
    "sample_stats" : {
      "total_sample_count" : 309,
      "unannotated_sample_count" : 308,
      "unreviewed_sample_count" : 0,
      "uncheck_sample_count" : 1,
      "sampled_sample_count" : 0,
      "rejected_sample_count" : 0,
      "accepted_sample_count" : 0,
      "auto_annotation_sample_count" : 0
    },
    "auto_sync_dataset" : true
  }, {
    "dataset_id" : "WxCREuCkBSAlQr9xrde",
    "task_id" : "iYZx7gScPUozOXner9k",
    "task_name" : "task-e63f",
    "status" : 1,
    "create_time" : 1606184400278,
    "update_time" : 1606184400278,
    "repetition" : 1,
    "workforces_config" : {
      "workforces" : [ {
        "workforce_id" : "q3ZFSwORu1ztKljDLYQ",
        "workforce_name" : "modelarts-team",
        "workers" : [ {
          "email" : "xxx@xxx.com",
          "worker_id" : "afdda13895bc66322ffbf36ae833bcf0",
          "role" : 0
        } ]
      } ]
    },
    "synchronize_data" : false,
    "synchronize_auto_labeling_data" : false,
    "workforce_stats" : {
      "workforce_count" : 1,
      "labeler_count" : 1,
      "reviewer_count" : 0
    },
    "sample_stats" : {
      "total_sample_count" : 317,
      "unannotated_sample_count" : 310,
      "unreviewed_sample_count" : 0,
      "uncheck_sample_count" : 0,
      "sampled_sample_count" : 0,
      "rejected_sample_count" : 0,
      "accepted_sample_count" : 7,
      "auto_annotation_sample_count" : 0
    },
    "checking_task_desc" : {
      "checking_task_id" : "onSbri2oqYOmDjDyW17",
      "action" : 0,
      "overwrite_last_result" : false
    },
    "auto_sync_dataset" : true
  } ]
}

Status Codes

Status Code

Description

200

OK

401

Unauthorized

403

Forbidden

404

Not Found

Error Codes

See Error Codes.