Querying Details About a Team Labeling Task

Function

This API is used to query the details about a team labeling task.

Debugging

You can debug this API through automatic authentication in or use the SDK sample code generated by API Explorer.

URI

GET /v2/{project_id}/datasets/{dataset_id}/workforce-tasks/{workforce_task_id}

Table 1 Path Parameters

Parameter

Mandatory

Type

Description

dataset_id

Yes

String

Dataset ID.

project_id

Yes

String

Project ID. For details about how to obtain a project ID, see Obtaining a Project ID and Name.

workforce_task_id

Yes

String

ID of a team labeling task.

Table 2 Query Parameters

Parameter

Mandatory

Type

Description

label_stats

No

Boolean

Whether to return label statistics. Options:

  • true: Return label statistics.

  • false: Do not return label statistics. (Default value)

sample_stats

No

Boolean

Whether to return sample statistics. Options:

  • true: Return sample statistics.

  • false: Do not return sample statistics. (Default value)

workforce_stats

No

Boolean

Whether to return personnel statistics. Options:

  • true: Return personnel statistics.

  • false: Do not return personnel statistics. (Default value)

Request Parameters

None

Response Parameters

Status code: 200

Table 3 Response body parameters

Parameter

Type

Description

auto_sync_dataset

Boolean

Whether to automatically synchronize the result of a team labeling task to the dataset. Options:

  • true: Automatically synchronize the result of a team labeling task to the dataset.

  • false: Do not automatically synchronize the result of a team labeling task to the dataset.

check_rate

Double

Acceptance ratio of a team labeling task.

checking_task_desc

WorkforceSamplingTaskDesc object

Details about the current acceptance task of a team labeling task.

create_time

Long

Time when a labeling task is created.

dataset_id

String

Dataset ID.

description

String

Description of a labeling task.

label_stats

Array of LabelStats objects

Label statistics of a labeling task.

pass_rate

Double

Acceptance pass rate of a team labeling task.

repetition

Integer

Number of persons who label each sample in a team labeling task. The minimum value is 1.

sample_search_conditions

Array of SearchCondition objects

Sample search conditions when creating a task.

sample_stats

SampleStats object

Sample statistics of a labeling task.

score

Double

Average acceptance score of a team labeling task.

status

Integer

Status of a team labeling task. Options:

  • 0: Starting. The owner or manager is starting the task, and files to be labeled are still being allocated. The owner, manager, annotator, and reviewer can view the task list. Since the task allocation has not been completed, no other task can be started.

  • 1: Running. The annotator or reviewer performs labeling or reviews the task, and the owner performs acceptance. For example, new files need to be allocated again if auto labeling files are added or synchronized, or unlabeled files are imported.

  • 2: Accepting. The owner initiates an acceptance task, but the acceptance has not been completed. In this state, a new acceptance task can be started only after the current one is completed.

  • 3: Accepted. The team labeling task has been completed.

  • **4: Rejected. The manager starts the task again. In this state, samples need to be labeled and the task must be reviewed again.

  • 5: Synchronizing acceptance result. This status is added when an acceptance task changes to be asynchronous. In this state, no new acceptance task can be initiated and the current acceptance task cannot be continued. In the task name area, the system displays a message indicating that the acceptance result is being synchronized.

  • 6: Created. The task has been created by the owner but has not been started. Only the owner and manager can view the task list.

  • 7: Acceptance sampling. This status is added when an acceptance task changes to be asynchronous. In this state, no new acceptance task can be initiated and the current acceptance task cannot be continued. In the task name area, the system displays a message indicating that acceptance sampling is in progress.

synchronize_auto_labeling_data

Boolean

Whether to synchronize the auto labeling result of a team labeling task. Options:

  • true: Synchronize the results to be confirmed to team members after auto labeling is complete.

  • false: Do not synchronize the auto labeling results. (Default value)

synchronize_data

Boolean

Whether to synchronize the added data of a team labeling task. Options:

  • true: Upload files, synchronize data sources, and synchronize imported unlabeled files to team members.

  • false: Do not synchronize the added data. (Default value)

task_id

String

ID of a labeling task.

task_name

String

Name of a labeling task.

update_time

Long

Time when a labeling task is updated.

version_id

String

Version ID of the dataset associated with a labeling task.

worker_stats

Array of WorkerTask objects

Labeling progress statistics on team labeling task members.

workforce_stats

WorkforceStats object

Statistics on team labeling task members.

workforces_config

WorkforcesConfig object

Team labeling task information: Tasks can be assigned by the team administrator or a specified team.

Table 4 WorkforceSamplingTaskDesc

Parameter

Type

Description

action

Integer

Acceptance operations. Options:

  • 0: Pass all samples.

  • 1: Reject all samples.

  • 4: Pass single-accepted samples and unaccepted samples.

  • 5: Reject single-rejected samples and unaccepted samples.

checking_stats

CheckTaskStats object

Real-time report of acceptance tasks.

checking_task_id

String

ID of the current acceptance task.

overwrite_last_result

Boolean

Whether to use the acceptance result to overwrite the labeled result if a sample has been labeled during acceptance. Options:

  • true: Overwrite the labeled result.

  • false: Do not overwrite the labeled result. (Default value)

total_stats

CheckTaskStats object

Overall report of historical acceptance tasks.

Table 5 CheckTaskStats

Parameter

Type

Description

accepted_sample_count

Integer

Accepted samples.

checked_sample_count

Integer

Checked samples.

pass_rate

Double

Pass rate of samples.

rejected_sample_count

Integer

Rejected samples.

sampled_sample_count

Integer

Number of sampled samples.

sampling_num

Integer

Samples of an acceptance task.

sampling_rate

Double

Sampling rate of an acceptance task.

score

String

Acceptance score.

task_id

String

ID of an acceptance task.

total_sample_count

Integer

Total samples.

total_score

Long

Total acceptance score.

unchecked_sample_count

Integer

Unchecked samples.

Table 6 LabelStats

Parameter

Type

Description

attributes

Array of LabelAttribute objects

Multi-dimensional attribute of a label. For example, if the label is music, attributes such as style and artist may be included.

count

Integer

Number of labels.

name

String

Label name.

property

LabelProperty object

Basic attribute key-value pair of a label, such as color and shortcut keys.

sample_count

Integer

Number of samples containing the label.

type

Integer

Label type. Options:

  • 0: image classification

  • 1: object detection

  • 3: image segmentation

  • 100: text classification

  • 101: named entity recognition

  • 102: text triplet relationship

  • 103: text triplet entity

  • 200: sound classification

  • 201: speech content

  • 202: speech paragraph labeling

  • 600: video labeling

Table 7 LabelAttribute

Parameter

Type

Description

default_value

String

Default value of a label attribute.

id

String

Label attribute ID.

name

String

Label attribute name.

type

String

Label attribute type. Options:

  • text: text

  • select: single-choice drop-down list

values

Array of LabelAttributeValue objects

List of label attribute values.

Table 8 LabelAttributeValue

Parameter

Type

Description

id

String

Label attribute value ID.

value

String

Label attribute value.

Table 9 LabelProperty

Parameter

Type

Description

@modelarts:color

String

Default attribute: Label color, which is a hexadecimal code of the color. By default, this parameter is left blank. Example: #FFFFF0.

@modelarts:default_shape

String

Default attribute: Default shape of an object detection label (dedicated attribute). By default, this parameter is left blank. Options:

  • bndbox: rectangle

  • polygon: polygon

  • circle: circle

  • line: straight line

  • dashed: dotted line

  • point: point

  • polyline: polyline

@modelarts:from_type

String

Default attribute: Type of the head entity in the triplet relationship label. This attribute must be specified when a relationship label is created. This parameter is used only for the text triplet dataset.

@modelarts:rename_to

String

Default attribute: The new name of the label.

@modelarts:shortcut

String

Default attribute: Label shortcut key. By default, this parameter is left blank. For example: D.

@modelarts:to_type

String

Default attribute: Type of the tail entity in the triplet relationship label. This attribute must be specified when a relationship label is created. This parameter is used only for the text triplet dataset.

Table 10 SearchCondition

Parameter

Type

Description

coefficient

String

Filter by coefficient of difficulty.

frame_in_video

Integer

A frame in the video.

hard

String

Whether a sample is a hard sample. Options:

  • 0: non-hard sample

  • 1: hard sample

import_origin

String

Filter by data source.

kvp

String

CT dosage, filtered by dosage.

label_list

SearchLabels object

Label search criteria.

labeler

String

Labeler.

metadata

SearchProp object

Search by sample attribute.

parent_sample_id

String

Parent sample ID.

sample_dir

String

Directory where data samples are stored (the directory must end with a slash (/)). Only samples in the specified directory are searched for. Recursive search of directories is not supported.

sample_name

String

Search by sample name, including the file name extension.

sample_time

String

When a sample is added to the dataset, an index is created based on the last modification time (accurate to day) of the sample on OBS. You can search for the sample based on the time. Options:

  • month: Search for samples added from 30 days ago to the current day.

  • day: Search for samples added from yesterday (one day ago) to the current day.

  • yyyyMMdd-yyyyMMdd: Search for samples added in a specified period (at most 30 days), in the format of Start date-End date. For example, 20190901-2019091501 indicates that samples generated from September 1 to September 15, 2019 are searched.

score

String

Search by confidence.

slice_thickness

String

DICOM layer thickness. Samples are filtered by layer thickness.

study_date

String

DICOM scanning time.

time_in_video

String

A time point in the video.

Table 11 SearchLabels

Parameter

Type

Description

labels

Array of SearchLabel objects

List of label search criteria.

op

String

If you want to search for multiple labels, op must be specified. If you search for only one label, op can be left blank. Options:

  • OR: OR operation

  • AND: AND operation

Table 12 SearchLabel

Parameter

Type

Description

name

String

Label name.

op

String

Operation type between multiple attributes. Options:

  • OR: OR operation

  • AND: AND operation

property

Map<String,Array<String>>

Label attribute, which is in the Object format and stores any key-value pairs. key indicates the attribute name, and value indicates the value list. If value is null, the search is not performed by value. Otherwise, the search value can be any value in the list.

type

Integer

Label type. Options:

  • 0: image classification

  • 1: object detection

  • 3: image segmentation

  • 100: text classification

  • 101: named entity recognition

  • 102: text triplet relationship

  • 103: text triplet entity

  • 200: sound classification

  • 201: speech content

  • 202: speech paragraph labeling

  • 600: video labeling

Table 13 SearchProp

Parameter

Type

Description

op

String

Relationship between attribute values. Options:

  • AND: AND relationship

  • OR: OR relationship

props

Map<String,Array<String>>

Search criteria of an attribute. Multiple search criteria can be set.

Table 14 WorkerTask

Parameter

Type

Description

create_time

Long

Time when a labeling team member's task is created.

dataset_id

String

ID of a dataset associated with a labeling team member's task.

dataset_type

Integer

Labeling type of a team member's task.

email

String

Email address of a labeling team member.

email_status

Integer

Email notification status of a labeling team member's labeling task. Options:

  • 0: The email has not been sent.

  • 1: The email format is incorrect.

  • 2: The email address is unreachable.

  • 3: The email has been sent.

last_notify_time

Long

Timestamp of the latest notification email sent to a labeling team member.

pass_rate

Double

Pass rate of task acceptance review for a labeling team member.

role

Integer

Role of a labeling team member.

sample_stats

SampleStats object

Sample statistics of a labeling team member's task.

score

Double

Average acceptance score of labeling team members' task samples.

task_id

String

Team labeling task ID associated with a member's task.

task_status

Integer

Task status of a labeling team member. Options:

  • 6: created

  • 0: starting

  • 1: running

  • 2: under acceptance

  • 3: approved, indicating the team labeling task is complete

  • 4: rejected, indicating that the task needs to be labeled and reviewed again

update_time

Long

Time when a labeling team member's task is updated.

worker_id

String

ID of a labeling team member.

workforce_task_name

String

Team labeling task name associated with a member's task.

Table 15 SampleStats

Parameter

Type

Description

accepted_sample_count

Integer

Number of samples accepted by the owner.

auto_annotation_sample_count

Integer

Number of samples to be confirmed after intelligent labeling.

deleted_sample_count

Integer

Number of deleted samples.

rejected_sample_count

Integer

Number of samples that failed to pass the owner acceptance.

sampled_sample_count

Integer

Number of samples that are to be accepted by the owner and sampled.

total_sample_count

Integer

Total number of samples.

unannotated_sample_count

Integer

Number of unlabeled samples.

uncheck_sample_count

Integer

Number of samples that have been approved by the reviewer and are to be accepted by the owner.

unreviewed_sample_count

Integer

Number of samples that have been labeled by the labeler but have not been reviewed by the reviewer.

Table 16 WorkforceStats

Parameter

Type

Description

labeler_count

Integer

Number of labeling persons.

reviewer_count

Integer

Number of reviewers.

workforce_count

Integer

Number of teams.

Table 17 WorkforcesConfig

Parameter

Type

Description

agency

String

Administrator

workforces

Array of WorkforceConfig objects

List of teams that execute labeling tasks.

Table 18 WorkforceConfig

Parameter

Type

Description

workers

Array of Worker objects

List of labeling team members.

workforce_id

String

ID of a labeling team.

workforce_name

String

Name of a labeling team. The value contains 0 to 1024 characters and does not support the following special characters: !<>=&"'

Table 19 Worker

Parameter

Type

Description

create_time

Long

Creation time.

description

String

Labeling team member description. The value contains 0 to 256 characters and does not support the following special characters: ^!<>=&"'

email

String

Email address of a labeling team member.

role

Integer

Role. Options:

  • 0: labeling personnel

  • 1: reviewer

  • 2: team administrator

  • 3: dataset owner

status

Integer

Current login status of a labeling team member. Options:

  • 0: The invitation email has not been sent.

  • 1: The invitation email has been sent but the user has not logged in.

  • 2: The user has logged in.

  • 3: The labeling team member has been deleted.

update_time

Long

Update time.

worker_id

String

ID of a labeling team member.

workforce_id

String

ID of a labeling team.

Example Requests

Querying Details About a Team Labeling Task

GET https://{endpoint}/v2/{project_id}/datasets/{dataset_id}/workforce-tasks/{workforce_task_id}

Example Responses

Status code: 200

OK

{
  "dataset_id" : "WxCREuCkBSAlQr9xrde",
  "task_id" : "iYZx7gScPUozOXner9k",
  "task_name" : "task-e63f",
  "status" : 1,
  "create_time" : 1606184400278,
  "update_time" : 1606184400278,
  "repetition" : 1,
  "workforces_config" : {
    "workforces" : [ {
      "workforce_id" : "q3ZFSwORu1ztKljDLYQ",
      "workers" : [ {
        "email" : "xxx@xxx.com",
        "worker_id" : "afdda13895bc66322ffbf36ae833bcf0",
        "role" : 0
      } ]
    } ]
  },
  "synchronize_data" : false,
  "synchronize_auto_labeling_data" : false,
  "workforce_stats" : {
    "workforce_count" : 1,
    "labeler_count" : 1,
    "reviewer_count" : 0
  },
  "sample_stats" : {
    "total_sample_count" : 317,
    "unannotated_sample_count" : 310,
    "unreviewed_sample_count" : 0,
    "uncheck_sample_count" : 0,
    "sampled_sample_count" : 0,
    "rejected_sample_count" : 0,
    "accepted_sample_count" : 7,
    "auto_annotation_sample_count" : 0
  },
  "checking_task_desc" : {
    "checking_task_id" : "onSbri2oqYOmDjDyW17",
    "action" : 0,
    "overwrite_last_result" : false
  },
  "auto_sync_dataset" : true,
  "worker_stats" : [ {
    "email" : "xxx@xxx.com",
    "worker_id" : "afdda13895bc66322ffbf36ae833bcf0",
    "role" : 0,
    "task_id" : "iYZx7gScPUozOXner9k",
    "workforce_task_name" : "task-e63f",
    "dataset_id" : "WxCREuCkBSAlQr9xrde",
    "sample_stats" : {
      "total_sample_count" : 317,
      "unannotated_sample_count" : 310,
      "unreviewed_sample_count" : 0,
      "uncheck_sample_count" : 0,
      "sampled_sample_count" : 0,
      "rejected_sample_count" : 0,
      "accepted_sample_count" : 7,
      "auto_annotation_sample_count" : 0
    },
    "create_time" : 1606184400278,
    "update_time" : 1606184795050,
    "email_status" : 3,
    "last_notify_time" : 0
  } ]
}

Status Codes

Status Code

Description

200

OK

401

Unauthorized

403

Forbidden

404

Not Found

Error Codes

See Error Codes.