Configuring a GaussDB Data Source¶
Scenario¶
This section describes how to add a GaussDB JDBC data source on the HSConsole page.
Prerequisites¶
The domain name of the cluster where the data source is located must be different from the HetuEngine cluster domain name.
The cluster where the data source is located and the HetuEngine cluster nodes can communicate with each other.
A HetuEngine compute instance has been created.
Procedure¶
Log in to FusionInsight Manager as a HetuEngine administrator and choose Cluster > Services > HetuEngine. The HetuEngine service page is displayed.
In the Basic Information area on the Dashboard page, click the link next to HSConsole WebUI. The HSConsole page is displayed.
Choose Data Source.
Click Add Data Source. Configure parameters on the Add Data Source page.
Configure Basic Information. For details, see Table 1.
Table 1 Basic Information¶ Parameter
Description
Example Value
Name
Name of the data source to be connected.
The value can contain only letters, digits, and underscores (_) and must start with a letter.
gaussdb_1
Data Source Type
Type of the data source to be connected. Choose JDBC > GAUSSDB-A.
GAUSSDB-A
Description
Description of the data source.
The value can contain only letters, digits, commas (,), periods (.), underscores (_), spaces, and line breaks.
-Configure parameters in the GAUSSDB-A Configuration area. For details, see Table 2.
Table 2 GAUSSDB-A Configuration¶ Parameter
Description
Example Value
Driver
The default value is gaussdba.
gaussdba
JDBC URL
JDBC URL for connecting to GaussDB A. The format is as follows:
jdbc:postgresql://CN service IP address:Port number/Database name
Username
Username for connecting to the GaussDB data source.
Change the value based on the username being connected with the data source.
Password
Password for connecting to the GaussDB data source.
Change the value based on the username and password for connecting to the data source.
GaussDB User Information Configuration
Configure multiple GaussDB usernames and passwords in the format of dataSourceUser and password key-value pairs.
-HetuEngine-GaussDB User Mapping Configuration
Multiple HetuEngine accounts are configured in the format of hetuUser and dataSourceUser key-value pairs, corresponding to one of the users configured in the GaussDB User Information Configuration area. When different HetuEngine users are used to access GaussDB, different GaussDB usernames and passwords can be used.
-Configure the GaussDB data source user information. For details, see Table 3.
GaussDB User Information Configuration and HetuEngine-GaussDB User Mapping Configuration must be used together. When HetuEngine is connected to the GaussDB data source, HetuEngine users can have the same permissions of the mapped GaussDB data source user through mapping. Multiple HetuEngine users can correspond to one GaussDB user.
Table 3 GaussDB User Information Configuration¶ Parameter
Description
Example Value
Data Source User
Data Source User
The value can contain only letters, digits, underscores (_), hyphens (-), and periods (.), and must start with a letter or underscore (_). The minimum length is 2 characters and the maximum length is 100 characters.
If the data source user is set to gaussuser1, a HetuEngine user mapped to gaussuser1 must exist.
For example, create hetuuser1 and map it to gaussuser1.
Password
User authentication password of the corresponding data source.
-Configure the HetuEngine-GaussDB user mapping. For details, see Table 4.
Table 4 HetuEngine-GaussDB User Mapping Configuration¶ Parameter
Description
Example Value
HetuEngine User
HetuEngine username.
The value can contain only letters, digits, underscores (_), hyphens (-), and periods (.), and must start with a letter or underscore (_). The minimum length is 2 characters and the maximum length is 100 characters.
hetuuser1
Data Source User
Data source user.
The value can contain only letters, digits, underscores (_), hyphens (-), and periods (.), and must start with a letter or underscore (_). The minimum length is 2 characters and the maximum length is 100 characters.
gaussuser1 (data source user configured in Table 3)
Modify custom configurations.
You can click Add to add custom configuration parameters. Configure custom parameters of the GaussDB data source. For details, see Table 5.
Table 5 Custom parameters of the GaussDB data source¶ Parameter
Description
Example Value
use-connection-pool
Whether to use the JDBC connection pool.
true
jdbc.connection.pool.maxTotal
Maximum number of connections in the JDBC connection pool.
8
jdbc.connection.pool.maxIdle
Maximum number of idle connections in the JDBC connection pool.
8
jdbc.connection.pool.minIdle
Minimum number of idle connections in the JDBC connection pool.
0
jdbc.pushdown-enabled
true: SQL statements can be pushed down to the data source for execution.
false: SQL statements are not pushed down to the data source for execution. As a result, more network and computing resources are consumed.
true
jdbc.pushdown-module
The push-down function should be enabled in advance.
DEFAULT: No operator is pushed down.
BASE_PUSHDOWN: Only operators such as Filter, Aggregation, Limit, TopN, and Projection are pushed down.
FULL_PUSHDOWN: All supported operators are pushed down.
DEFAULT
source-encoding
GaussDB data source encoding mode.
UTF-8
multiple-cnn-enabled
Whether to use the GaussDB multi-CN configuration. To use it, ensure that the JDBC connection pool is disabled and the JDBC URL format is as follows: jdbc:postgresql://host:port/database,jdbc:postgresql://host:port/database,jdbc:postgresql://host:port/database.
false
parallel-read-enabled
Whether to use the parallel data read function.
If the parallel data read function is enabled, the actual number of splits is determined based on the node distribution and the value of max-splits.
Multiple connections to the data source will be created for parallel read operations. The dependent data source should support the load.
false
split-type
Type of the parallel data read function.
NODE: The degree of parallelism (DOP) is categorized based on the GaussDB data source DataNodes.
PARTITION: The DOP is categorized based on table partitions.
INDEX: The DOP is categorized based on table indexes.
NODE
max-splits
Maximum degree of parallelism.
5
use-copymanager-for-insert
Whether to use CopyManager for batch import.
false
unsupported-type-handling
If the connector does not support the data of a certain type, convert it to VARCHAR.
After the CONVERT_TO_VARCHAR parameter is configured, the data of BIT VARYING, CIDR, MACADDR, INET, OID, REGTYPE, REGCONFIG and POINT types are converted to the varchar type during query and data of these types can only be read.
The default value is IGNORE, indicating that unsupported types will be not displayed in the result.
CONVERT_TO_VARCHAR
max-bytes-in-a-batch-for-copymanager-in-mb
Maximum volume of data imported by CopyManager in a batch, in MB.
10
You can click Delete to delete custom configuration parameters.
Click OK.
Important
The UPDATE and DELETE syntaxes do not support filtering clauses containing cross-catalog conditions, for example, UPDATE mppdb.table SET column1=value WHERE column2 IN (SELECT column2 from hive.table).
To use the DELETE syntax, set jdbc.pushdown-enabled to true and unsupported-type-handling to CONVERT_TO_VARCHAR.
The DELETE syntax does not support filtering clauses containing subqueries, for example, DELETE FROM mppdb.table WHERE column IN (SELECT column FROM mppdb.table1).
HetuEngine supports a maximum precision of 38 digits for GaussDB data sources of the NUMBER data type