To DWS¶

If the destination link of a job is a DWS link, configure the destination job parameters based on Table 1.

**Table 1** Parameter description¶
Category	Parameter	Description	Example Value
Basic parameters	Schema/Tablespace	Name of the database to which data will be written. The schema can be automatically created. Click the icon next to the text box to select a schema or tablespace.	schema
	Auto Table Creation	This parameter is displayed only when the source is a relational database. The options are as follows: Non-auto creation: CDM will not automatically create a table. Auto creation: If the destination database does not contain the table specified by Table Name, CDM will automatically create the table. If the table specified by Table Name already exists, no table is created and data is written to the existing table. Deletion before creation: CDM deletes the table specified by Table Name, and then creates the table again. Field Mapping in Automatic Table Creation on DWS describes the field mapping between the DWS tables created by CDM and source tables.	Non-auto creation
	Table Name	Name of the table to which data will be written. Click the icon next to the text box. The dialog box for selecting the table is displayed. This parameter can be configured as a macro variable of date and time and a path name can contain multiple macro variables. When the macro variable of date and time works with a scheduled job, the incremental data can be synchronized periodically.	table
	Compress Data	Whether to compress data when data is imported to DWS and Auto creation is selected	No
	Storage Mode	When data is imported to DWS and Auto Creation is selected, you can specify the data storage mode: Row-based: Row-based storage. It is used for point queries (index-based simple queries with fewer return records), or the scenario that requires a large number of addition, deletion, and modification operations. Column-based: Column-based storage. It is used for statistical analysis queries (group and join scenarios) or ad hoc queries (query conditions are uncertain and indexes can hardly be used to scan row-based tables).	Row-based
	Import Mode	Mode for importing data to DWS In COPY mode, the source data is copied to the DataNode of DWS after passing through the management node. In UPSERT mode, if a primary key or unique constraint conflict occurs, other data columns, except the primary key and unique constraint column, are updated.	COPY
	Clear Data Before Import	Whether to clear the data in the destination table before data import. The options are as follows: Do not clear: The data in the destination table is not cleared before data import. The imported data is just added to the table. Clear all data: All data is cleared from the destination table before data import. Clear part of data: Part of the data in the destination table is cleared before data import. If you select Clear part of data, you must configure WHERE Clause to specify which part will be deleted.	Clear part of data
	WHERE Clause	If Clear Data Before Import is set to Clear part of data, data in the destination table will be deleted based on the WHERE clause after the configuration is complete and before the import starts.	age > 18 and age <= 60
	Constraint Conflict Handling	Mode for handling conflicts in data migration insert into: When a primary key or unique index conflict occurs, data cannot be written and will become dirty data. replace into: When a primary key or unique index conflict occurs, the original row is deleted and a new row is inserted to replace all the fields in the original row. on duplicate key update: When a primary key or unique index conflict occurs in a row in the destination table, the data columns except the unique constraint column in this row are updated.	insert into
	Loader Threads	Number of threads started in each loader. A larger number allows more concurrent write operations. Note This parameter is unavailable if Constraint Conflict Handling is set to replace into or on duplicate key update.	1
Advanced parameters	Import to Staging Table	If you set this parameter to Yes, the transaction mode is enabled. CDM automatically creates a temporary table and imports data to the temporary table. After the data is imported successfully, it is migrated to the destination table in transaction mode. If the import fails, the destination table is rolled back to the state before the job starts. The default value is No, indicating that CDM directly imports the data to the destination table. In this case, if the job fails to be executed, the data that has been imported to the destination table will not be rolled back automatically. Note If you select Clear part of data or Clear all data for Clear Data Before Import, CDM does not roll back the deleted data in transaction mode.	No
	Extending field length	When Auto creation is selected, the length of the character fields can be extended to three times the original length and then written to the destination table. If the encoding types of the source and destination databases are different, but the character fields in the source and destination tables are the same, errors may occur during data migration due to character length difference. When a character field containing Chinese characters is imported to DWS, the length of the character field must be automatically increased by three times. If a job fails to be executed and an error message similar to value too long for type character varying exists in the log when you import Chinese characters to DWS, you can enable this function to solve the problem. Note When this function is enabled, some fields consume three times the storage space of the user.	No
	Use NOT NULL Constraint	If you choose to create a target table automatically and specify the NOT NULL constraint, keep the NOT NULL constraints of the source and target tables consistent.	Yes
	Prepare for Data Import	The SQL statement that is first executed before a task is executed. Currently, only one SQL statement can be executed in wizard mode.	create temp table
	Complete Statement After Data Import	The SQL statement that is executed after a task is executed. Currently, only one SQL statement can be executed.	merge into

Field Mapping in Automatic Table Creation on DWS¶

Figure 1 describes the field mapping between DWS tables created by CDM and source tables. For example, if you use CDM to migrate the Oracle database to DWS, CDM automatically creates a table on DWS and maps the NUMBER(3,0) field of the Oracle database to the SMALLINT field of DWS.

**Figure 1** Field mapping in automatic table creation¶

last updated: 2024-11-13 13:47 UTC - commit: db9c60a6c98025320fa32706a5e58cd02cf7d7f1