• MapReduce Service

mrs
  1. Help Center
  2. MapReduce Service
  3. User Guide
  4. Cluster Operation Guide
  5. Bootstrap Actions
  6. Preparing the Bootstrap Action Script

Preparing the Bootstrap Action Script

Currently, bootstrap actions support Linux shell scripts only. Script files must end with .sh.

Uploading the Installation Packages and Files to an OBS Bucket

Before compiling a script, you need to upload all required installation packages, configuration packages, and relevant files to the OBS bucket in the same region. Because networks of different regions are isolated from each other, MRS VMs cannot download OBS files from other regions.

How to Specify a Script to Download Files from the OBS Bucket

You can specify the file to be downloaded from OBS in the script. If you upload files to a private bucket, you need to run the hadoop fs command to download the files. The following example shows that the s3a://yourbucket/myfile.tar.gz file will be downloaded to the local host and decompressed to the /your-dir directory.

#!/bin/bash
source /opt/client/bigdata_env;hadoop fs -Dfs.s3a.endpoint=<obs-endpoint> -Dfs.s3a.access.key=<your-ak> -Dfs.s3a.secret.key=<your-sk> -copyToLocal s3a://yourbucket/myfile.tar.gz ./
mkdir -p /<your-dir>
tar -zxvf myfile.tar.gz -C /<your-dir>
NOTE:
  • The Hadoop client has been preinstalled on the MRS node. You can run the hadoop fs command to download or upload data from or to OBS.
  • Regions and Endpoints lists the obs-endpoint of each region.
  • Sample Scripts shows that the installation packages have been uploaded to the public readable OBS bucket. Therefore, you can run the curl command in the sample script to download the installation packages.

Uploading the Script to an OBS Bucket

After script compilation, upload the script to the OBS bucket in the same region. At the time you specify, each node in the cluster downloads the script from OBS and executes the script as user root.