Using the HDFS Client¶
Scenario¶
This section describes how to use the HDFS client in an O&M scenario or service scenario.
Prerequisites¶
The client has been installed.
For example, the installation directory is /opt/hadoopclient. The client directory in the following operations is only an example. Change it to the actual installation directory.
Service component users are created by the administrator as required. In security mode, machine-machine users need to download the keytab file. A human-machine user needs to change the password upon the first login. (This operation is not required in normal mode.)
Using the HDFS Client¶
Log in to the node where the client is installed as the client installation user.
Run the following command to go to the client installation directory:
cd /opt/hadoopclient
Run the following command to configure environment variables:
source bigdata_env
If the cluster is in security mode, run the following command to authenticate the user. In normal mode, user authentication is not required.
kinit Component service user
Run the HDFS Shell command. Example:
hdfs dfs -ls /
Common HDFS Client Commands¶
The following table lists common HDFS client commands.
For more commands, see https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CommandsManual.html#User_Commands.
Command | Description | Example |
---|---|---|
hdfs dfs -mkdir Folder name | Used to create a folder. | hdfs dfs -mkdir /tmp/mydir |
hdfs dfs -ls Folder name | Used to view a folder. | hdfs dfs -ls /tmp |
hdfs dfs -put Local file on the client node | Used to upload a local file to a specified HDFS path. | hdfs dfs -put /opt/test.txt /tmp Upload the /opt/tests.txt file on the client node to the /tmp directory of HDFS. |
hdfs dfs -get Specified file on HDFS Specified path on the client node | Used to download the HDFS file to the specified local path. | hdfs dfs -get /tmp/test.txt /opt/ Download the /tmp/test.txt file on HDFS to the /opt path on the client node. |
hdfs dfs -rm -r -f Specified folder on HDFS | Used to delete a folder. | hdfs dfs -rm -r -f /tmp/mydir |