Real-Time Synchronization¶
Real-time synchronization refers to the real-time flow of key service data from sources to destinations while consistency of data can be ensured. It is different from migration. Migration means moving your overall database from one platform to another. Synchronization refers to the continuous flow of data between different services.
Supported Database Types¶
DRS supports real-time synchronization between databases of various types, and many-to-one synchronization.
Synchronization Direction | Data Flow | Source DB | Destination DB | Destination DB Type |
---|---|---|---|---|
To the cloud | MySQL->MySQL |
| RDS MySQL DB instances |
|
To the cloud | MySQL-> GaussDB distributed |
| GaussDB distributed |
|
To the cloud | MySQL -> GaussDB(for MySQL) primary/standby |
| GaussDB(for MySQL) primary/standby |
|
To the cloud | PostgreSQL->PostgreSQL |
| RDS PostgreSQL DB instances |
|
To the cloud | Oracle->PostgreSQL |
| RDS PostgreSQL DB instances |
|
To the cloud | DDM->DDM | DDM | DDM | Clusters |
From the cloud | MySQL->MySQL | RDS MySQL DB instances |
|
|
From the cloud | MySQL->Kafka | RDS MySQL DB instances |
|
|
From the cloud | GaussDB distributed -> MySQL | GaussDB distributed |
|
|
Self-built -> Self-built | MySQL->MySQL |
|
|
|
Self-built -> Self-built | MySQL->Kafka |
|
|
|
Synchronization Methods¶
DRS supports three synchronization modes: full synchronization, incremental synchronization, and full+incremental synchronization.
Full synchronization: All objects and data in non-system databases are synchronized to the destination database at a time. This mode is applicable to scenarios where service interruption is acceptable.
Incremental synchronization: Through log parsing, DRS replicates incremental data to keep sources and destinations in sync.
Full+Incremental synchronization: DRS allows you to synchronize data in real time. After a full synchronization initializes the destination database, an incremental synchronization parses logs to ensure data consistency between the source and destination databases.
Synchronization Direction | Data Flow | Incremental | Full | Full+Incremental | One-way/Two-way Sync |
---|---|---|---|---|---|
To the cloud | MySQL->MySQL | Supported | Not supported | Supported | One-way sync |
To the cloud | MySQL-> GaussDB distributed | Not supported | Not supported | Supported | One-way sync |
To the cloud | MySQL -> GaussDB(for MySQL) primary/standby | Supported | Not supported | Supported | One-way sync |
To the cloud | PostgreSQL->PostgreSQL | Supported | Supported | Supported | One-way sync |
To the cloud | DDM->DDM | Not supported | Not supported | Supported | One-way sync |
To the cloud | Oracle->PostgreSQL | Not supported | Supported | Supported | One-way sync |
From the cloud | MySQL->MySQL | Supported | Not supported | Supported | One-way sync |
From the cloud | MySQL->Kafka | Supported | Not supported | Not supported | One-way sync |
From the cloud | GaussDB distributed -> MySQL | Supported | Not supported | Not supported | One-way sync |
Self-built -> Self-built | MySQL->MySQL | Supported | Not supported | Supported | One-way sync |
Self-built -> Self-built | MySQL->Kafka | Supported | Not supported | Not supported | One-way sync |
Database Versions¶
Note
Data cannot be synchronized from a newer version database to an older version database.
Synchronization Direction | Data Flow | Source Database Version | Destination DB Version |
---|---|---|---|
To the cloud | MySQL->MySQL |
|
|
To the cloud | MySQL-> GaussDB distributed |
|
|
To the cloud | MySQL -> GaussDB(for MySQL) primary/standby |
| GaussDB(for MySQL)-MySQL 8.0 |
To the cloud | PostgreSQL->PostgreSQL |
|
|
To the cloud | DDM->DDM | Based on the live network | Based on the live network |
To the cloud | Oracle->PostgreSQL |
|
|
From the cloud | MySQL->MySQL |
|
|
From the cloud | MySQL->Kafka |
| Kafka 0.11 or later |
From the cloud | GaussDB distributed -> MySQL | GaussDB 1.3 |
|
Self-built -> Self-built | MySQL->MySQL |
|
|
Self-built -> Self-built | MySQL->Kafka |
| Kafka 0.11 or later |
Network Types¶
DRS supports real-time synchronization through a Virtual Private Cloud (VPC), Virtual Private Network (VPN), Direct Connect, or public network. Table 4 lists the application scenarios of each network type and required preparations.
Network Type | Application Scenario | Preparations |
---|---|---|
VPC | Synchronization between cloud databases in the same region |
|
VPN | Synchronization from on-premises databases to cloud databases or between cloud databases across regions | Establish a VPN connection between your local data center and the VPC that hosts the destination database. Before synchronization, ensure that the VPN network is accessible. |
Direct Connect | Synchronization from on-premises databases to cloud databases or between cloud databases across regions | Use a dedicated network connection to connect your data center to VPCs. |
Public network | Synchronization from on-premises or external cloud databases to the destination databases. | To ensure network connectivity between the source and destination databases, perform the following operations:
|
Synchronization Direction | Data Flow | VPC | Public Network | VPN or Direct Connect |
---|---|---|---|---|
To the cloud | MySQL->MySQL | Supported | Supported | Supported |
To the cloud | MySQL-> GaussDB distributed | Supported | Supported | Supported |
To the cloud | MySQL -> GaussDB(for MySQL) primary/standby | Supported | Supported | Supported |
To the cloud | PostgreSQL->PostgreSQL | Supported | Supported | Supported |
To the cloud | DDM->DDM | Supported | Supported | Supported |
To the cloud | Oracle->PostgreSQL | Supported | Supported | Supported |
From the cloud | MySQL->MySQL | Supported | Supported | Supported |
From the cloud | MySQL->Kafka | Supported | Supported | Supported |
From the cloud | GaussDB distributed -> MySQL | Not supported | Supported | Supported |
Self-built -> Self-built | MySQL->MySQL | Not supported | Supported | Supported |
Self-built -> Self-built | MySQL->Kafka | Supported | Supported | Supported |
Supported Synchronization Objects¶
DRS allows you to synchronize different objects. The following table lists the supported objects.
Synchronization Direction | Data Flow | Table-level | Database-level | Importing an Object File |
---|---|---|---|---|
To the cloud | MySQL->MySQL | Supported | Supported | Supported |
To the cloud | MySQL-> GaussDB distributed | Supported | Not supported | Not supported |
To the cloud | MySQL -> GaussDB(for MySQL) primary/standby | Supported | Supported | Supported |
To the cloud | PostgreSQL->PostgreSQL | Supported | Supported | Supported |
To the cloud | DDM->DDM | Supported | Supported | Not supported |
To the cloud | Oracle->PostgreSQL | Supported | Not supported | Supported |
From the cloud | MySQL->MySQL | Supported | Supported | Not supported |
From the cloud | MySQL->Kafka | Supported | Supported | Supported |
From the cloud | GaussDB distributed -> MySQL | Supported | Not supported | Not supported |
Self-built -> Self-built | MySQL->MySQL | Supported | Supported | Not supported |
Self-built -> Self-built | MySQL->Kafka | Supported | Supported | Supported |
Advanced Features¶
DRS supports multiple features to ensure successful data synchronization.
Feature | Description |
---|---|
Synchronization level | DRS supports database- and table-level synchronization.
|
Mapping object names | Allows the names of synchronization objects (including databases, schemas, tables, and columns) in the source database to be different from those in the destination database. If the synchronization objects in source and destination databases have different names, you can map the source object name to the destination one. The following objects can be mapped: databases, schemas and tables. |
Dynamically adding or deleting synchronization objects | During data synchronization, you can add or delete synchronization objects as required. |
Conflict policy | DRS uses primary key or unique key conflict policies to ensure that tables with primary key or unique constraints in the source database can be synchronized to the destination database as expected. The following conflict policies are supported:
Ignore and overwrite: Synchronization stability is prioritized, so tasks will not be interrupted as data conflicts occur. Report error: Data quality is prioritized. Any data conflicts are not allowed, so once a conflict occurs, the synchronization task fails and an error is reported. You need to manually find the cause of the fault. If the task is in the failed state for a long time, the storage space may be used up and the task cannot be restored. |
Structure synchronization | DRS does not provide data structure synchronization as an independent function during real-time synchronization. Instead, it directly synchronizes data and structures to the destination database. |