Accessing the Spark Web UI

The Spark web UI is used to view the running status of Spark applications. Google Chrome is recommended for better user experience.

Spark has two web UIs.

  • Spark UI: used to display the status of running applications.

    The UI includes the following parts: Jobs, Stages, Storage, Environment, Executors, SQL, and JDBC/ODBC Server. The Streaming application has the Streaming tab in addition to the preceding parts.

  • History Server UI: used to display the status of Spark applications that are complete or incomplete.

    The UI includes the application ID, application name, start time, end time, execution time, and owner information.

Spark UI

  1. Access the component management page.

    • For versions earlier than MRS 1.9.2, log in to MRS Manager and choose Services.

    • For versions earlier than MRS 3.x, click the cluster name to go to the cluster details page and choose Components.

      Note

      If the Components tab is unavailable, complete IAM user synchronization first. (On the Dashboard page, click Synchronize on the right side of IAM User Sync to synchronize IAM users.)

    • For MRS 3.x or later, log in to FusionInsight Manager. For details, see Accessing FusionInsight Manager (MRS 3.x or Later). Choose Cluster > Name of the desired cluster > Services.

  2. Select Yarn. In the Yarn Summary area, click ResourceManager in ResourceManager Web UI to access the web UI.

  3. Locate the Spark application. Click ApplicationMaster in the last column of the application information. The Spark UI is displayed.

    **Figure 1** ApplicationMaster

    Figure 1 ApplicationMaster

    **Figure 2** Spark UI

    Figure 2 Spark UI

History Server

  1. Access the component management page.

    • For versions earlier than MRS 1.9.2, log in to MRS Manager and choose Services.

    • For versions earlier than MRS 3.x, click the cluster name to go to the cluster details page and choose Components.

      Note

      If the Components tab is unavailable, complete IAM user synchronization first. (On the Dashboard page, click Synchronize on the right side of IAM User Sync to synchronize IAM users.)

    • For MRS 3.x or later, log in to FusionInsight Manager. For details, see Accessing FusionInsight Manager (MRS 3.x or Later). Choose Cluster > Name of the desired cluster > Services.

  2. Select Spark. In the Spark Summary area, click JobHistory corresponding to Spark Web UI to access the web UI.

    **Figure 3** Spark History Server

    Figure 3 Spark History Server

last updated: 2024-05-10 00:32