• MapReduce Service

mrs
  1. Help Center
  2. MapReduce Service
  3. User Guide
  4. Cluster Operation Guide
  5. Managing Active Clusters
  6. Performing Auto Scaling for a Cluster

Performing Auto Scaling for a Cluster

Auto scaling can automatically adjust computing resources based on service requirements and the policies preset by users, so the number of Task nodes can increase or decrease with service load changes, ensuring stable service running.

Background

Auto scaling rules:
  • A user can set a maximum of five rules for expanding or shrinking a cluster, respectively.
  • The system judges rules set by the user in sequence and cluster expansion rules take priorities over cluster shrink rules. Place rules according to their importance degrees and put the most important rule in the front to prevent the rules from being repeatedly triggered due to the unexpected result of cluster expansion or shrink.
  • Comparison factors are Greater thanGreater than or equal toLess than, and Less than or equal to.
  • Cluster expansion or shrink can be triggered only after the configured metric threshold is reached for consecutive 5n (the default value of n is 1) minutes.
  • After each cluster expansion or shrink, there is cooling time. The default cooling time is 20 minutes and the minimum cooling time is 0 minutes.
  • In each cluster expansion or shrinking, at least one node and at most 100 nodes can be added or reduced.

Procedure

  1. Log in to the MRS management console.
  2. Click in the upper-left corner on the management console and select Region and Project.
  3. Choose Clusters > Active Clusters, select a running cluster, and click its name to switch to the cluster information page.
  4. Click  to unfold cluster details and click  behind Task Node Auto Scaling. The Auto Scaling page is displayed.
  5. Configure auto scaling rules.

    You can adjust the number of nodes to configure auto scaling rules. Node quantity adjustment affects prices. Adjust nodes with caution.

    Figure 1 Auto Scaling
    • Auto Scaling: indicates whether to enable auto scaling. Auto scaling is disabled by default. After you enable it, you can configure the following parameters.
    • Nodes: Enter the minimum and maximum number of nodes. This value range is 0 to 500 and applies to all expansion and shrinking rules.
    • Auto Scaling Rule: To enable Auto Scaling, configure expansion or shrinking rules.

      Configuration procedure:

      1. Select Expand or Shrink.
      2. Click Add Rule. The Add Rule page is displayed.
        Figure 2 Add Rule
      3. Configure the parameters Rule NameIfLastAddCooling Time.
      4. Click OK.

        You can view the rules you configured in the Expand or Shrink area on the Auto Scaling page.

    • Select I agree to authorize MRS to expand or shrink nodes based on the above policy..

  6. Click OK.

Related Information

When adding rules, you can refer to Table 1 to configure auto scaling metrics.

Table 1 Auto scaling metrics

Cluster Type

Metric Name

Type

Description

Streaming cluster

StormSlotAvailable

Integer

Number of available Storm slots

Value range: 0 to 2147483647

StormSlotAvailablePercentage

Percentage

Percentage of available Storm slots, that is, the proportion of available slots to total slots

Value range: 0 to 100

StormSlotUsed

Integer

Number of the used Storm slots

Value range: 0 to 2147483647

StormSlotUsedPercentage

Percentage

Percentage of the used Storm slots, that is, the proportion of the used slots to total slots

Value range: 0 to 100

Analysis cluster

YARNAppPending

Integer

Number of pending tasks on YARN

Value range: 0 to 2147483647

YARNAppPendingRatio

Ratio

Ratio of pending tasks on YARN, that is, the ratio of pending tasks to running tasks on Yarn

Value range: 0 to 2147483647

YARNAppRunning

Integer

Number of running tasks on YARN

Value range: 0 to 2147483647

YARNContainerAllocated

Integer

Number of containers allocated to YARN

Value range: 0 to 2147483647

YARNContainerPending

Integer

Number of pending containers on YARN

Value range: 0 to 2147483647

YARNContainerPendingRatio

Ratio

Ratio of pending containers on YARN, the ratio of pending containers to running containers on Yarn

Value range: 0 to 2147483647

YARNCPUAllocated

Integer

Number of virtual CPUs (vCPUs) allocated to YARN

Value range: 0 to 2147483647

YARNCPUAvailable

Integer

Number of available vCPUs on YARN

Value range: 0 to 2147483647

YARNCPUAvailablePercentage

Percentage

Percentage of available vCPUs on YARN, that is, the proportion of available vCPUs to total vCPUs

Value range: 0 to 100

YARNCPUPending

Integer

Number of pending vCPUs on YARN

Value range: 0 to 2147483647

YARNMemoryAllocated

Integer

Memory allocated to YARN. The unit is MB.

Value range: 0 to 2147483647

YARNMemoryAvailable

Integer

Available memory on YARN. The unit is MB.

Value range: 0 to 2147483647

YARNMemoryAvailablePercentage

Percentage

Percentage of available memory on YARN, that is, the proportion of available memory to total memory on YARN

Value range: 0 to 100

YARNMemoryPending

Integer

Pending memory on YARN

Value range: 0 to 2147483647

NOTE:

When the value type is percentage or ratio in Table 1, the valid value can be accurate to percentile. The percentage metric value is a decimal value with a percent sign (%) removed. For example, 16.80 represents 16.80%.