Scale the TiFlash Cluster

This document describes how to scale in and out nodes in the TiFlash cluster.

Scale out a TiFlash node

The following example shows how to scale out a TiFlash node if you deploy TiFlash on the 192.168.1.1 node.

  1. Edit the inventory.ini file to add the TiFlash node information:

    Currently, you can only add IP but not domain name.

    [tiflash_servers] 192.168.1.1
  2. Edit the hosts.ini file to add the node information:

    [servers] 192.168.1.1 [all:vars] username = tidb ntp_server = pool.ntp.org
  3. Initialize the new node:

    • Configure the SSH mutual trust and sudo rules on the Control Machine:

      ansible-playbook -i hosts.ini create_users.yml -l 192.168.1.1 -u root -k
    • Install the NTP service on the target machine:

      ansible-playbook -i hosts.ini deploy_ntp.yml -u tidb -b
    • Initialize the node on the target machine:

      ansible-playbook bootstrap.yml -l 192.168.1.1
  4. Deploy the new node:

    ansible-playbook deploy.yml -l 192.168.1.1
  5. Start the new node:

    ansible-playbook start.yml -l 192.168.1.1
  6. Update the configuration of Prometheus and restart it:

    ansible-playbook rolling_update_monitor.yml --tags=prometheus
  7. Access the monitoring platform using your browser, and view the status of the cluster and the new node.

Scale in a TiFlash node

The following example shows how to scale in a TiFlash node if you stop the TiFlash service on the 192.168.1.1 node.

  1. Take down the node to be scaled in. See Take a TiFlash node down for details.

  2. Check whether the node has been offline successfully using Grafana or pd-ctl (the offline process takes some time).

  3. After the store corresponding to TiFlash disappears, or the state_name becomes Tombstone, execute the following command to shutdown the TiFlash process:

    ansible-playbook stop.yml -l 192.168.1.1

    If the node still has other services and you want to stop TiFlash only, use the following command:

    ansible-playbook stop.yml -t tiflash -l 192.168.1.1
  4. Edit the inventory.ini and hosts.ini files to remove the node information.

  5. Update the configuration of Prometheus and restart it:

    ansible-playbook rolling_update_monitor.yml --tags=prometheus
  6. Access the monitoring platform using your browser, and view the status of the cluster.