• MapReduce Service

mrs
  1. Help Center
  2. MapReduce Service
  3. User Guide
  4. MRS Manager Operation Guide
  5. Alarm Reference
  6. ALM-14002 DataNode Disk Usage Exceeds the Threshold

ALM-14002 DataNode Disk Usage Exceeds the Threshold

Description

The system checks the DataNode disk usage every 30 seconds and compares it with the threshold. This alarm is generated when the value of Percentage of DataNode Capacity exceeds the threshold and is cleared when the value is less than or equal to the threshold.

Attribute

Alarm ID

Alarm Severity

Automatically Cleared

14002

Major

Yes

Parameters

Parameter

Description

ServiceName

Specifies the service for which the alarm is generated.

RoleName

Specifies the role for which the alarm is generated.

HostName

Specifies the host for which the alarm is generated.

Trigger Condition

Generates an alarm when the actual indicator value exceeds the specified threshold.

Impact on the System

The performance of writing data to HDFS is affected.

Possible Causes

  • The disk space configured for the HDFS cluster is insufficient.
  • Data skew occurs among DataNodes.

Procedure

  1. Check the cluster disk capacity.

    1. Log in to MRS Manager. On the Alarm page, check whether alarm ALM-14001 HDFS Disk Usage Exceeds the Threshold exists.
      • If yes, go to 1.b.
      • If no, go to 2.a.
    2. Follow the procedures in ALM-14001 HDFS Disk Usage Exceeds the Threshold to handle the alarm and check whether the alarm is cleared.
    3. Wait 5 minutes and check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to 2.a.

  2. Check the balance status of DataNodes.

    1. Use the client on the cluster node and run the hdfs dfsadmin -report command to view the value of DFS Used% on the DataNode that generated the alarm. Compare this value with those on other DataNodes and check whether the difference between the values is greater than 10.
    2. If data skew occurs, use the client on the cluster node and run the hdfs balancer -threshold 10 command.
    3. Wait 5 minutes and check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to Step 3.

  3. Collect fault information.

    1. On MRS Manager, choose System > Export Log.
    2. Contact technical support engineers for help, detail see technical support.

Related Information

N/A