• MapReduce Service

mrs
  1. Help Center
  2. MapReduce Service
  3. User Guide
  4. Cluster Operation Guide
  5. Managing Jobs
  6. Submitting a Spark SQL Statement

Submitting a Spark SQL Statement

This section describes how to use Spark SQL. You can submit a Spark SQL statement to query and analyze data on the MRS management console page. To submit multiple statements, separate them from each other using semicolons (;).

Procedure

  1. Log in to the MRS management console.
  2. Click in the upper-left corner on the management console and select Region and Project.
  3. Choose Clusters > Active Clusters, select a running cluster, and click its name to switch to the cluster information page.
  4. Click Job Management and go to the Job Management tab page.
  5. Select Spark SQL. The Spark SQL job page is displayed.
  6. Enter the Spark SQL statement for table creation.

    When entering Spark SQL statements, ensure that they have no more than 10,000 characters.

    Syntax:

    CREATE [EXTERNAL] TABLE [IF NOT EXISTS] table_name [(col_name data_type [COMMENT col_comment], ...)] [COMMENT table_comment] [PARTITIONED BY (col_name data_type [COMMENT col_comment], ...)] [CLUSTERED BY (col_name, col_name, ...) [SORTED BY (col_name [ASC|DESC], ...)] INTO num_buckets BUCKETS] [ROW FORMAT row_format] [STORED AS file_format] [LOCATION hdfs_-path];

    Use either of the following methods to create a table:

    • Method 1: Create an src_data table and write data in every row. The data is stored in the /user/guest/input directory.

      create external table src_data(line string) row format delimited fields terminated by '\\n' stored as text file location '/user/guest/input/';

    • Method 2: Create an src_data table and load the data to the src_dada1 table.

      create table src_data1 (eid int, name String, salary String, destination String) row format delimited fields terminated by ',' ;

      load data in path '/tttt/test.txt' into table src_data1;

    NOTE:

    The data from OBS cannot be loaded to the created tables in method 2.

  7. Enter the Spark SQL statement for table query.

    Syntax:

    SELECT col_name FROM table_name;

    Example:

    select * from src_data;

  8. Enter the Spark SQL statement for table deletion.

    Syntax:

    DROP TABLE [IF EXISTS] table_name;

    Example:

    drop table src_data;

  9. Click Check to check the statement correctness.
  10. Click Submit.

    After submitting Spark SQL statements, you can check whether the execution is successful in Last Execution Result and view detailed execution results in Last Query Result Set.