Deploy a DM Cluster Offline Using TiUP

This document describes how to deploy a DM cluster offline using TiUP.

Step 1: Prepare the TiUP offline component package

  • Install the TiUP package manager online.

    1. Install the TiUP tool:

      curl --proto '=https' --tlsv1.2 -sSf https://tiup-mirrors.pingcap.com/install.sh | sh
    2. Redeclare the global environment variables:

      source .bash_profile
    3. Confirm whether TiUP is installed:

      which tiup
  • Pull the mirror using TiUP

    1. Pull the needed components on a machine that has access to the Internet:

      # You can modify ${version} to the needed version. tiup mirror clone tidb-dm-${version}-linux-amd64 --os=linux --arch=amd64 \ --dm-master=${version} --dm-worker=${version} --dmctl=${version} \ --alertmanager=v0.17.0 --grafana=v4.0.3 --prometheus=v4.0.3 \ --tiup=v$(tiup --version|grep 'tiup'|awk -F ' ' '{print $1}') --dm=v$(tiup --version|grep 'tiup'|awk -F ' ' '{print $1}')

      The command above creates a directory named tidb-dm-${version}-linux-amd64 in the current directory, which contains the component package managed by TiUP.

    2. Pack the component package by using the tar command and send the package to the control machine in the isolated environment:

      tar czvf tidb-dm-${version}-linux-amd64.tar.gz tidb-dm-${version}-linux-amd64

      tidb-dm-${version}-linux-amd64.tar.gz is an independent offline environment package.

Step 2: Deploy the offline TiUP component

After sending the package to the control machine of the target cluster, install the TiUP component by running the following command:

# You can modify ${version} to the needed version. tar xzvf tidb-dm-${version}-linux-amd64.tar.gz sh tidb-dm-${version}-linux-amd64/local_install.sh source /home/tidb/.bash_profile

The local_install.sh script automatically executes the tiup mirror set tidb-dm-${version}-linux-amd64 command to set the current mirror address to tidb-dm-${version}-linux-amd64.

To switch the mirror to another directory, manually execute the tiup mirror set <mirror-dir> command. If you want to switch back to the official mirror, execute tiup mirror set https://tiup-mirrors.pingcap.com.

Step 3: Edit the initialization configuration file

You need to edit the cluster initialization configuration file according to different cluster topologies.

For the full configuration template, refer to the TiUP configuration parameter template. Create a configuration file topology.yaml. In other combined scenarios, edit the configuration file as needed according to the templates.

The configuration of deploying three DM-masters, three DM-workers, and one monitoring component instance is as follows:

--- global: user: "tidb" ssh_port: 22 deploy_dir: "/home/tidb/dm/deploy" data_dir: "/home/tidb/dm/data" # arch: "amd64" master_servers: - host: 172.19.0.101 - host: 172.19.0.102 - host: 172.19.0.103 worker_servers: - host: 172.19.0.101 - host: 172.19.0.102 - host: 172.19.0.103 monitoring_servers: - host: 172.19.0.101 grafana_servers: - host: 172.19.0.101 alertmanager_servers: - host: 172.19.0.101

Step 4: Execute the deployment command

tiup dm deploy dm-test ${version} ./topology.yaml --user root [-p] [-i /home/root/.ssh/gcp_rsa]

In the above command:

  • The name of the deployed DM cluster is dm-test.
  • The version of the DM cluster is ${version}. You can view the latest versions supported by TiUP by running tiup list dm-master.
  • The initialization configuration file is topology.yaml.
  • --user root: Log in to the target machine through the root key to complete the cluster deployment, or you can use other users with ssh and sudo privileges to complete the deployment.
  • [-i] and [-p]: optional. If you have configured login to the target machine without password, these parameters are not required. If not, choose one of the two parameters. [-i] is the private key of the root user (or other users specified by --user) that has access to the target machine. [-p] is used to input the user password interactively.
  • TiUP DM uses the embedded SSH client. If you want to use the SSH client native to the control machine system, edit the configuration according to using the system's native SSH client to connect to the cluster.

At the end of the output log, you will see Deployed cluster `dm-test` successfully. This indicates that the deployment is successful.

Step 5: Check the clusters managed by TiUP

tiup dm list

TiUP supports managing multiple DM clusters. The command above outputs information of all the clusters currently managed by TiUP, including the name, deployment user, version, and secret key information:

Name User Version Path PrivateKey ---- ---- ------- ---- ---------- dm-test tidb ${version} /root/.tiup/storage/dm/clusters/dm-test /root/.tiup/storage/dm/clusters/dm-test/ssh/id_rsa

Step 6: Check the status of the deployed DM cluster

To check the status of the dm-test cluster, execute the following command:

tiup dm display dm-test

Expected output includes the instance ID, role, host, listening port, and status (because the cluster is not started yet, so the status is Down/inactive), and directory information of the dm-test cluster.

Step 7: Start the cluster

tiup dm start dm-test

If the output log includes Started cluster `dm-test` successfully, the start is successful.

Step 8: Verify the running status of the cluster

Check the DM cluster status using TiUP:

tiup dm display dm-test

If the Status is Up in the output, the cluster status is normal.