Important

You are viewing the archived documentation of TiDB, which no longer receives updates. It is recommended that you use the latest LTS version of the TiDB database.

Deploy TiDB Operator in Kubernetes

This document describes how to deploy TiDB Operator in Kubernetes.

Prerequisites

Before deploying TiDB Operator, make sure the following items are installed on your machine:

Kubernetes >= v1.12
DNS addons
PersistentVolume
RBAC enabled (optional)
Helm version >= 2.11.0 and < 2.16.4

Note

Although TiDB Operator can use network volume to persist TiDB data, this can affect performance a lot because TiDB stores multiple replicas itself. So it is highly recommended to set up local volume for better performance.
Network volumes in a multi-availability zone setup require Kubernetes v1.12 or higher version. It is recommended to use networked volumes to store backup data in tidb-bakup chart.

Deploy Kubernetes cluster

TiDB Operator runs in Kubernetes cluster. You can refer to the document of how to set up Kubernetes to set up a Kubernetes cluster. Make sure that the Kubernetes version is v1.12 or higher. If you are using AWS, GKE or local machines, here are quick-start tutorials:

If you are deploying in a different environment, a proper DNS addon must be installed in the Kubernetes cluster. You can follow the official documentation to set up a DNS addon.

TiDB Operator uses Persistent Volume to persist the data of TiDB cluster (including the database, monitoring data, backup data), so the Kubernetes cluster must provide at least one kind of persistent volume. For better performance, it is recommended to use local SSD disk as the volumes. Follow this step to auto-provision local persistent volumes.

It is suggested to enable RBAC in the Kubernetes cluster. Otherwise, you need to set rbac.create to false in the values.yaml of both tidb-operator and tidb-cluster charts.

Because TiDB uses many file descriptors by default, the worker node and its Docker daemon's ulimit values must be greater than or equal to 1048576.

Configure the ulimit value of the work node. See How to set ulimit values.
```
sudo vim /etc/security/limits.conf
```
Set the nofile values of soft and hard of the root account to be greater than or equal to 1048576.
Configure the ulimit value of the Docker service.
```
sudo vim /etc/systemd/system/docker.service
```
Set LimitNOFILE to be greater than or equal to 1048576.
Note
You need to explicitly set LimitNOFILE to 1048576 or a larger value rather than the default infinity. Because of a bug in systemd, the value of infinity is 65536 in some systemd versions.

Install Helm

Refer to Use Helm to install Helm and configre it with the official PingCAP chart Repo.

Configure local persistent volume

Prepare local volumes

Refer to Local PV Configuration to set up local persistent volumes in your Kubernetes cluster.

Install TiDB Operator

TiDB Operator uses Custom Resource Definition (CRD) to extend Kubernetes. Therefore, to use TiDB Operator, you must first create the TidbCluster custom resource type, which is a one-time job in your Kubernetes cluster.

kubectl apply -f https://raw.githubusercontent.com/pingcap/tidb-operator/master/manifests/crd.yaml && \
kubectl get crd tidbclusters.pingcap.com

After TidbCluster custom resource type is created, install TiDB Operator in your Kubernetes cluster.

Get the values.yaml file of the tidb-operator chart you want to install.
```
mkdir -p /home/tidb/tidb-operator && \
helm inspect values pingcap/tidb-operator --version=<chart-version> > /home/tidb/tidb-operator/values-tidb-operator.yaml
```
Note
<chart-version> represents the chart version of TiDB Operator. For example, v1.0.0. You can view the currently supported versions by running the helm search -l tidb-operator command.

Install TiDB Operator.

helm install pingcap/tidb-operator --name=tidb-operator --namespace=tidb-admin --version=<chart-version> -f /home/tidb/tidb-operator/values-tidb-operator.yaml && \
kubectl get po -n tidb-admin -l app.kubernetes.io/name=tidb-operator

Customize TiDB Operator

To customize TiDB Operator, modify /home/tidb/tidb-operator/values-tidb-operator.yaml. The rest sections of the document use values.yaml to refer to /home/tidb/tidb-operator/values-tidb-operator.yaml

TiDB Operator contains two components:

tidb-controller-manager
tidb-scheduler

These two components are stateless and deployed via Deployment. You can customize resource limit, request, and replicas in the values.yaml file.

After modifying values.yaml, run the following command to apply this modification:

helm upgrade tidb-operator pingcap/tidb-operator --version=<chart-version> -f /home/tidb/tidb-operator/values-tidb-operator.yaml