• MapReduce Service

mrs
  1. Help Center
  2. MapReduce Service
  3. API Reference
  4. APIs
  5. Auto Scaling APIs
  6. Configuring an Auto Scaling Rule

Configuring an Auto Scaling Rule

Function

This API is used to configure an auto scaling rule. This API is incompatible with Sahara.

The API used for cluster creation and job execution can also be used to creation an auto scaling rule.

URI

  • Format:

    POST /v1.1/{project_id}/autoscaling-policy/{cluster_id}

  • Parameter description
    Table 1 URI parameter description

    Parameter

    Mandatory or Not

    Description

    project_id

    Yes

    Project ID. For details on how to obtain the project ID, see Obtaining a Project ID.

    cluster_id

    Yes

    Cluster ID

Request

  • Example
    {
      "node_group":"task_node_default_group",
      "auto_scaling_policy": {
          "auto_scaling_enable": true,
          "min_capacity": 1,
          "max_capacity": "3",
          "rules": [{
             "name": "default-expand-1",
             "adjustment_type": "scale_out",
             "cool_down_minutes": 5,
             "scaling_adjustment": 1,
             "trigger": {
                "metric_name": "YARNMemoryAvailablePercentage",
                "metric_value": "25",
                "comparison_operator": "LT",
                "evaluation_periods": 10
                }
              },
             {
              "name": "default-shrink-1",
              "adjustment_type": "scale_in",
              "cool_down_minutes": 5,
              "scaling_adjustment": 1,
              "trigger": {
                 "metric_name": "YARNMemoryAvailablePercentage",
                 "metric_value": "70",
                 "comparison_operator": "GT",
                 "evaluation_periods": 10
                  }
               }]
         }
    }
    NOTE:

    A new auto scaling rule will overwrite the auto scaling rule saved in the original database. If you want to modify the original rule, query the original rule first, modify the rule, and then submit a modification task. For details, see Viewing Cluster Details.

  • Parameter description
    Table 2 node_group parameter description

    Parameter

    Mandatory or Not

    Type

    Description

    node_group

    Yes

    String

    Node type that is applicable to auto scaling rules. Currently, only Task nodes are supported. The parameter value is task_node_default_group.

    Table 3 auto_scaling_policy parameter description

    Parameter

    Mandatory or Not

    Type

    Description

    auto_scaling_enable

    Yes

    Boolean

    Whether the auto scaling rule is enabled

    min_capacity

    Yes

    Integer

    Minimum number of nodes left in the node group

    Value range: 0 to 500

    max_capacity

    Yes

    Integer

    Maximum number of nodes in the node group

    Value range: 0 to 500

    rules

    Yes

    List

    List of auto scaling rules. For details, see Table 4.

    Table 4 rules parameter description

    Parameter

    Mandatory or Not

    Type

    Description

    name

    Yes

    String

    Name of an auto scaling rule

    It contains only 1 to 64 letters, digits, hyphens (-), and underscores (_).

    Rule names must be unique in a node group.

    description

    No

    String

    Description about an auto scaling rule

    Contains a maximum of 1024 characters.

    adjustment_type

    Yes

    String

    Elastic scaling rule adjustment type. Possible values are:

    • scale_out: cluster expansion
    • scale_in: cluster shrink

    cool_down_minutes

    Yes

    Integer

    Cluster cooling time after an auto scaling rule is triggered, when no auto scaling operation is performed. The unit is minute.

    Value range: 0 to 10080. There are 10080 minutes in a week.

    scaling_adjustment

    Yes

    Integer

    Number of nodes that can be adjusted once

    Value range: 0 to 100

    trigger

    Yes

    Trigger

    Rule triggering condition. For details, see Table 5.

    Table 5 trigger parameter description

    Parameter

    Mandatory or Not

    Type

    Description

    metric_name

    Yes

    String

    Metric name.

    This triggering condition makes a judgment according to the value of the metric corresponding to the name.

    Contains a maximum of 64 characters.

    Table 6 lists the supported metric names.

    metric_value

    Yes

    String

    Metric value.

    Metric threshold to trigger a rule. This parameter value can be an integer or number with two decimal places only.

    Table 6 provides values types and value ranges of metric_value.

    comparison_operator

    Yes

    String

    Metric judgment logic operator, including:

    • LT: less than
    • GT: greater than
    • LTOE: less than or equal to
    • GTOE: greater than or equal to

    evaluation_periods

    Yes

    Integer

    Number of consecutive five-minute periods, during which a metric threshold is reached

    Value range: 1 to 288

    Table 6 Auto scaling metrics

    Cluster Type

    Metric Name

    Type

    Description

    Streaming cluster

    StormSlotAvailable

    Integer

    Number of available Storm slots

    Value range: 0 to 2147483647

    StormSlotAvailablePercentage

    Percentage

    Percentage of available Storm slots, that is, the proportion of available slots to total slots

    Value range: 0 to 100

    StormSlotUsed

    Integer

    Number of the used Storm slots

    Value range: 0 to 2147483647

    StormSlotUsedPercentage

    Percentage

    Percentage of the used Storm slots, that is, the proportion of the used slots to total slots

    Value range: 0 to 100

    Analysis cluster

    YARNAppPending

    Integer

    Number of pending tasks on YARN

    Value range: 0 to 2147483647

    YARNAppPendingRatio

    Ratio

    Ratio of pending tasks on YARN, that is, the ratio of pending tasks to running tasks on Yarn

    Value range: 0 to 2147483647

    YARNAppRunning

    Integer

    Number of running tasks on YARN

    Value range: 0 to 2147483647

    YARNContainerAllocated

    Integer

    Number of containers allocated to YARN

    Value range: 0 to 2147483647

    YARNContainerPending

    Integer

    Number of pending containers on YARN

    Value range: 0 to 2147483647

    YARNContainerPendingRatio

    Ratio

    Ratio of pending containers on YARN, the ratio of pending containers to running containers on Yarn

    Value range: 0 to 2147483647

    YARNCPUAllocated

    Integer

    Number of virtual CPUs (vCPUs) allocated to YARN

    Value range: 0 to 2147483647

    YARNCPUAvailable

    Integer

    Number of available vCPUs on YARN

    Value range: 0 to 2147483647

    YARNCPUAvailablePercentage

    Percentage

    Percentage of available vCPUs on YARN, that is, the proportion of available vCPUs to total vCPUs

    Value range: 0 to 100

    YARNCPUPending

    Integer

    Number of pending vCPUs on YARN

    Value range: 0 to 2147483647

    YARNMemoryAllocated

    Integer

    Memory allocated to YARN. The unit is MB.

    Value range: 0 to 2147483647

    YARNMemoryAvailable

    Integer

    Available memory on YARN. The unit is MB.

    Value range: 0 to 2147483647

    YARNMemoryAvailablePercentage

    Percentage

    Percentage of available memory on YARN, that is, the proportion of available memory to total memory on YARN

    Value range: 0 to 100

    YARNMemoryPending

    Integer

    Pending memory on YARN

    Value range: 0 to 2147483647

    NOTE:

    When the value type is percentage or ratio in Table 6, the valid value can be accurate to percentile. The percentage metric value is a decimal value with a percent sign (%) removed. For example, 16.80 represents 16.80%.

Response

  • Example:

    { "result": "succeeded" }

  • Parameter description
    Table 7 Response parameter description

    Parameter

    Mandatory or Not

    Type

    Description

    result

    Yes

    String

    Operation result

    • succeeded: operation succeeded

Status Code

Table 8 describes the status code of this API.

Table 8 Status code

Status Code

Description

200

The cluster is successfully created.

For the description about error status codes, see section Status Codes.