Destination Link Configurations of Loader Jobs¶
Overview¶
When Loader jobs save data to different storage locations, a destination link needs to be selected and the link properties need to be configured.
obs-connector¶
Parameter | Description |
|---|---|
Bucket Name | OBS file system for storing final data. |
Output Directory | Directory for storing final data in the file system. A directory must be specified. |
File Format | Loader supports the following file formats of data stored in OBS:
|
Line Separator | Identifier of each line end of final data |
Field Separator | Identifier of each field end of final data |
Encoding Type | Text encoding type of final data. It takes effect on text files only. |
generic-jdbc-connector¶
Parameter | Description |
|---|---|
Schema Name | Name of the database storing final data |
Table | Name of the table saving final data |
ftp-connector or sftp-connector¶
Parameter | Description |
|---|---|
Output Directory | Directory for storing final data in the file server. A directory must be specified. |
File Format | Loader supports the following file formats of data stored in the file server:
|
Line Separator | Identifier of each line end of final data Note If FTP or SFTP serves as a destination link and File Format is set to BINARY_FILE, the value of Line Separator in the advanced properties is invalid. |
Field Separator | Identifier of each field end of final data Note If FTP or SFTP serves as a destination link and File Format is set to BINARY_FILE, the value of Field Separator in the advanced properties is invalid. |
Encoding Type | Text encoding type of final data. It takes effect on text files only. |
hbase-connector¶
Parameter | Description |
|---|---|
Table Name | Name of the HBase table saving final data. You can query and select it on the interface. |
Method | Data can be imported to an HBase table using either BULKLOAD or PUTLIST. |
Clear Data Before Import | Whether to clear data in the destination HBase table. Options are as follows:
|
hdfs-connector¶
Parameter | Description |
|---|---|
Output Directory | Directory for storing final data in HDFS. A directory must be specified. |
File Format | Loader supports the following file formats of data stored in HDFS:
|
Compression Codec | Compression mode used when a file is saved to HDFS. The following modes are supported: NONE, DEFLATE, GZIP, BZIP2, LZ4, and SNAPPY. |
Overwrite | How to process files in the output directory when files are imported to HDFS. Options are as follows:
|
Line Separator | Identifier of each line end of final data Note If HDFS serves as a destination link and File Format is set to BINARY_FILE, the value of Line Separator in the advanced properties is invalid. |
Field Separator | Identifier of each field end of final data Note If HDFS serves as a destination link and File Format is set to BINARY_FILE, the value of Field Separator in the advanced properties is invalid. |
hive-connector¶
Parameter | Description |
|---|---|
Database | Name of the Hive database storing final data. You can query and select it on the interface. |
Table | Name of the Hive table saving final data. You can query and select it on the interface. |