To HBase/CloudTable

Sample JSON File

"to-config-values": {
        "configs": [
          {
            "inputs": [
              {
                "name": "toJobConfig.table",
                "value": "rf_to"
              },
              {
                "name": "toJobConfig.storageType",
                "value": "PUTLIST"
              },
              {
                "name": "toJobConfig.columns",
                "value": "AA:AA&BB:BB&CC:CC&DD:DD"
              },
              {
                "name": "toJobConfig.rowKeyColumn",
                "value": "AA:AA"
              },
              {
                "name": "toJobConfig.isOverride",
                "value": "false"
              },
              {
                "name": "toJobConfig.isRowkeyRedundancy",
                "value": "false"
              },
              {
                "name": "toJobConfig.algorithm",
                "value": "NONE"
              },
              {
                "name": "toJobConfig.writeToWAL",
                "value": "true"
              },
              {
                "name": "toJobConfig.transType",
                "value": "false"
              }
            ],
            "name": "toJobConfig"
          }
        ]
      }

Parameter Description

Parameter	Mandatory	Type	Description
toJobConfig.table	Yes	String	Name of the table to which data is written. For example, TBL_EXAMPLE.
toJobConfig.storageType	Yes	Enumeration	Mode for writing data to an HBase table. The options are as follows: PUTLIST: Data is written using the putList() function.
toJobConfig.columns	No	String	Columns to be extracted. Use & to separate column numbers and : to separate column families and columns. For example, cf1:c1&cf2:c2.
toJobConfig.rowKeyColumn	Yes	String	Columns serve as rowkeys. Use & to separate column numbers and : to separate column families and columns. For example, cf1:c1&cf2:c2.
toJobConfig.isOverride	No	Boolean	Whether to clear data when data is imported in BULKLOAD mode. For example, true.
toJobConfig.delimiter	No	String	Delimiter used for separating columns when multiple columns are used as rowkeys. For example, vertical bars (|).
toJobConfig.isRowkeyRedundancy	No	Boolean	Whether to write rowkey data to the HBase column at the same time
toJobConfig.algorithm	No	Enumeration	Compression algorithm used when a new HBase table is created. The Snappy and GZ algorithms are supported. The default value is None.
toJobConfig.writeToWAL	No	Boolean	Whether to enable Write Ahead Log (WAL) of HBase. The options are as follows: Yes: If the HBase server breaks down after the function is enabled, you can replay the operations that have not been performed in WAL. No: If you set this parameter to No, the write performance is improved. However, if the HBase server breaks down, data may be lost.
toJobConfig.transType	No	Boolean	true: Data of the Short, Int, Long, Float, Double, and Decimal columns in the source database is converted into Byte[] arrays (binary) and written into HBase. Other types of data are written as character strings. If several types of data mentioned above are combined as rowkeys, they will be written as character strings. This function saves storage space. In specific scenarios, the rowkey distribution is evener. false: All types of data in the source database are written into HBase as character strings.

last updated: 2024-11-13 13:47 UTC - commit: db9c60a6c98025320fa32706a5e58cd02cf7d7f1