• MapReduce Service

mrs
  1. Help Center
  2. MapReduce Service
  3. User Guide
  4. MRS Manager Operation Guide
  5. Alarm Reference
  6. ALM-12017 Insufficient Disk Capacity

ALM-12017 Insufficient Disk Capacity

Description

The system checks the host disk usage every 30 seconds and compares it with the threshold. This alarm is generated when the host disk usage exceeds the specified threshold and is cleared when the host disk usage is less than or equal to the threshold.

Attribute

Alarm ID

Alarm Severity

Automatically Cleared

12017

Major

Yes

Parameters

Parameter

Description

ServiceName

Specifies the service for which the alarm is generated.

RoleName

Specifies the role for which the alarm is generated.

HostName

Specifies the host for which the alarm is generated.

PartitionName

Specifies the disk partition for which the alarm is generated.

Trigger Condition

Generates an alarm when the actual indicator value exceeds the specified threshold.

Impact on the System

Service processes become unavailable.

Possible Causes

The disk configuration does not meet service requirements. As a result, the disk usage reaches the upper limit.

Procedure

  1. Log in to MRS Manager and check whether the alarm threshold is appropriate.

    1. Choose System > Configure Alarm Threshold > Device > Disk > Disk Usage > Disk Usage and change the alarm threshold based on the actual disk usage.
    2. Wait 2 minutes and check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to Step 2.

  2. Check whether the disk is a system disk.

    1. In the alarm list on MRS Manager, locate the row that contains the alarm, and view its host name and disk partition information in the alarm details.
    2. Log in to the alarm node.
    3. Run the df -h command to check the system disk partition usage. Check whether the disk is mounted to the following directories based on the disk partition name obtained in 2.a//boot/home/opt/tmp/var/var/log/boot, and /srv/BigData.
      • If yes, the disk is a system disk. Go to 3.a.
      • If no, the disk is not a system disk. Go to 2.d.
    4. Run the df -h command to check the system disk partition usage. Determine the role of the disk based on the disk partition name obtained in 2.a.
    5. Check whether the disk is used by HDFS or Yarn.
      • If yes, expand the disk capacity for the Core node. Go to 2.f.
      • If no, go to Step 4.
    6. Wait 2 minutes and check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to Step 3.

  3. Check whether a large file is written to the disk.

    1. Run the find / -xdev -size +500M -exec ls -l {} \; command to view files larger than 500 MB on the node. Check whether these files are written to the disk.
    2. Process the large files and check whether the alarm is cleared after 2 minutes.
      • If yes, no further action is required.
      • If no, go to Step 4.
    3. Expand the disk capacity.
    4. Wait 2 minutes and check whether the alarm is cleared.
      • If yes, no further action is required.
      • If no, go to Step 4.

  4. Collect fault information.

    1. On MRS Manager, choose System > Export Log.
    2. Contact technical support engineers for help, detail see technical support.

Related Information

N/A