Data Migration Configuration File Overview

This document gives an overview of configuration files of DM (Data Migration).

DM process configuration files

  • inventory.ini: The configuration file of deploying DM using DM-Ansible. You need to edit it based on your machine topology. For details, see Edit the inventory.ini file to orchestrate the DM cluster.
  • dm-master.toml: The configuration file of running the DM-master process, including the topology information of the DM cluster and the corresponding relationship between the MySQL instance and DM-worker (must be one-to-one relationship). When you use DM-Ansible to deploy DM, dm-master.toml is generated automatically. Refer to DM-master Configuration File to see more details.
  • dm-worker.toml: The configuration file of running the DM-worker process, including the upstream MySQL instance configuration and the relay log configuration. When you use DM-Ansible to deploy DM, dm-worker.toml is generated automatically. Refer to DM-worker Configuration File to see more details.

DM migration task configuration

DM task configuration file

When you use DM-Ansible to deploy DM, you can find the following task configuration file template in <path-to-dm-ansible>/conf:

  • task.yaml.exmaple: The standard configuration file of the data migration task (a specific task corresponds to a task.yaml). For the introduction of the configuration file, see Task Configuration File.

Data migration task creation

You can perform the following steps to create a data migration task based on task.yaml.example:

  1. Copy task.yaml.example as your_task.yaml.
  2. Refer to the description in the Task Configuration File and modify the configuration in your_task.yaml.
  3. Create your data migration task using dmctl.

Important concepts

This section shows description of some important concepts.

ConceptDescriptionConfiguration File
source-idUniquely identifies a MySQL or MariaDB instance, or a replication group with the primary-secondary structure. The maximum length of source-id is 32.source_id of inventory.ini;
source-id of dm-master.toml;
source-id of task.yaml
DM-worker IDUniquely identifies a DM-worker (by the worker-addr parameter of dm-worker.toml)worker-addr of dm-worker.toml;
the -worker/-w flag of the dmctl command line