TiDB Docker Compose Deployment
This document describes how to quickly deploy a TiDB testing cluster with a single command using Docker Compose.
With Docker Compose, you can use a YAML file to configure application services in multiple containers. Then, with a single command, you can create and start all the services from your configuration.
Prerequisites
Make sure you have installed the following items on your machine:
Deploy TiDB using Docker Compose
Download
tidb-docker-compose
.git clone https://github.com/pingcap/tidb-docker-compose.gitChange the directory to tidb-docker-compose and get the latest TiDB Docker Images:
cd tidb-docker-compose && docker-compose pullStart the TiDB cluster:
docker-compose up -dUse the MySQL client to connect to TiDB to read and write data:
mysql -h 127.0.0.1 -P 4000 -u root
Monitor the cluster
After you have successfully deployed a TiDB cluster, you can now monitor the TiDB cluster using one of the following methods:
- Use Grafana to view the status of the cluster via http://localhost:3000 with the default account name and password:
admin
andadmin
. - Use TiDB-Vision, a cluster visualization tool, to see data transfer and load-balancing inside your cluster via http://localhost:8010.
Customize the cluster
After the deployment is completed, the following components are deployed by default:
- 3 PD instances, 3 TiKV instances, 1 TiDB instance
- Monitoring components: Prometheus, Pushgateway, Grafana
- Data visualization component: tidb-vision
To customize the cluster, you can edit the docker-compose.yml
file directly. It is recommended to generate docker-compose.yml
using the Helm template engine, because manual editing is tedious and error-prone.
Install Helm.
Helm can be used as a template rendering engine. Make sure your Helm version >= 2.9.0 and < 3.0.0.
To use Helm, you only need to download its binary file:
curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bashFor macOS, you can also install Helm using the following command in Homebrew:
brew install helm@2 brew link --force helm@2Download
tidb-docker-compose
.git clone https://github.com/pingcap/tidb-docker-compose.gitCustomize the cluster.
cd tidb-docker-compose vim compose/values.yaml # custom the cluster size, docker image, port mapping and so onYou can modify the configuration in
values.yaml
, such as the cluster size, TiDB image version, and so on.tidb-vision is the data visualization interface of the TiDB cluster, used to visually display the PD scheduling on TiKV data. If you do not need this component, comment out the
tidbVision
section.For PD, TiKV, TiDB and tidb-vision, you can build Docker images from GitHub source code or local files for development and testing.
- To build PD, TiKV or TiDB images from the locally compiled binary file, you need to comment out the
image
field and copy the compiled binary file to the correspondingpd/bin/pd-server
,tikv/bin/tikv-server
,tidb/bin/tidb-server
. - To build the tidb-vision image from local, you need to comment out the
image
field and copy the tidb-vision project totidb-vision/tidb-vision
.
- To build PD, TiKV or TiDB images from the locally compiled binary file, you need to comment out the
Generate the
docker-compose.yml
file.helm template compose > generated-docker-compose.ymlCreate and start the cluster using the generated
docker-compose.yml
file.docker-compose -f generated-docker-compose.yml pull # Get the latest Docker images docker-compose -f generated-docker-compose.yml up -dAccess the cluster.
mysql -h 127.0.0.1 -P 4000 -u rootAccess the Grafana monitoring interface:
- Default address: http://localhost:3000
- Default account name: admin
- Default password: admin
If tidb-vision is enabled, you can access the cluster data visualization interface: http://localhost:8010.
Access the Spark shell and load TiSpark
Insert some sample data to the TiDB cluster:
$ docker-compose exec tispark-master bash
$ cd /opt/spark/data/tispark-sample-data
$ mysql -h tidb -P 4000 -u root < dss.ddl
After the sample data is loaded into the TiDB cluster, you can access the Spark shell using docker-compose exec tispark-master /opt/spark/bin/spark-shell
.
$ docker-compose exec tispark-master /opt/spark/bin/spark-shell
...
Spark context available as 'sc' (master = local[*], app id = local-1527045927617).
Spark session available as 'spark'.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.1.1
/_/
Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_172)
Type in expressions to have them evaluated.
Type :help for more information.
scala> import org.apache.spark.sql.TiContext
...
scala> val ti = new TiContext(spark)
...
scala> ti.tidbMapDatabase("TPCH_001")
...
scala> spark.sql("select count(*) from lineitem").show
+--------+
|count(1)|
+--------+
| 60175|
+--------+
You can also access Spark with Python or R using the following commands:
docker-compose exec tispark-master /opt/spark/bin/pyspark
docker-compose exec tispark-master /opt/spark/bin/sparkR
For more details about TiSpark, see TiSpark Quick Start Guide.
Here is a 5-minute tutorial for macOS users that shows how to spin up a standard TiDB cluster using Docker Compose on your local computer.