ALM-43001 Spark2x Service Unavailable

Description

The system checks the Spark2x service status every 300 seconds. This alarm is generated when the Spark2x service is unavailable.

This alarm is cleared when the Spark2x service recovers.

Attribute

Alarm ID

Alarm Severity

Auto Clear

43001

Critical

Yes

Parameters

Name

Meaning

Source

Specifies the cluster for which the alarm is generated.

ServiceName

Specifies the service for which the alarm is generated.

RoleName

Specifies the role for which the alarm is generated.

HostName

Specifies the host for which the alarm is generated.

Impact on the System

The Spark tasks submitted by users fail to be executed.

Possible Causes

  • The KrbServer service is abnormal.

  • The LdapServer service is abnormal.

  • ZooKeeper is abnormal.

  • HDFS is abnormal.

  • Yarn is abnormal.

  • The corresponding Hive service is abnormal.

  • The Spark2x assembly package is abnormal.

Procedure

If the alarm is abnormal Spark2x assembly packet, the Spark packet is abnormal. Wait for about 10 minutes. The alarm is automatically cleared.

Check whether service unavailability alarms exist in services that Spark2x depends on.

  1. On FusionInsight Manager, choose O&M. In the navigation pane on the left, choose Alarm > Alarms.

  2. Check whether the following alarms exist in the alarm list:

    • ALM-25500 KrbServer Service Unavailable

    • ALM-25000 LdapServer Service Unavailable

    • ALM-13000 ZooKeeper Service Unavailable

    • ALM-14000 HDFS Service Unavailable

    • ALM-18000 Yarn Service Unavailable

    • ALM-16004 Hive Service Unavailable

    Note

    If the multi-instance function is enabled for the cluster and multiple Spark2x services are installed, check the Spark2x service for which the alarm is generated based on the value of ServiceName in location information and check whether the Hive service is faulty. Spark2x corresponds to Hive, spark2x1 corresponds to Hive1, and other services follow the same rule.

    • If yes, go to 3.

    • If no, go to 4.

  3. Handle the alarms based on the troubleshooting methods provided in the alarm help.

    After the alarm is cleared, wait a few minutes and check whether the alarm GuardianService Unavailable is cleared.

    • If yes, no further action is required.

    • If no, go to 4.

Collect fault information.

  1. On FusionInsight Manager, choose O&M > Log > Download.

  2. In the Service area, select the following nodes of the desired cluster. (Hive is the specific Hive service determined based on ServiceName in the alarm location information).

    • KrbServer

    • LdapServer

    • ZooKeeper

    • HDFS

    • Yarn

    • Hive

  3. Click image1 in the upper right corner, and set Start Date and End Date for log collection to 10 minutes ahead of and after the alarm generation time, respectively. Then, click Download.

  4. Contact O&M personnel and provide the collected logs.

Alarm Clearing

This alarm is automatically cleared after the fault is rectified.

last updated: 2024-05-08 03:04