TiDB Self-Managed
Learning Center
Contact Us
Sign InStart for Free

TiDB 4.0.9 Release Notes

Release date: December 21, 2020

TiDB version: 4.0.9

Compatibility Changes

  • TiDB

    • Deprecate the enable-streaming configuration item #21055

  • TiKV

    • Reduce I/O and mutex contention when encryption at rest is enabled. The change is backwardly incompatible. If users need to downgrade the cluster to a version earlier than v4.0.9, security.encryption.enable-file-dictionary-log must be disabled and TiKV must be restarted before the downgrade. #9195

New Features

  • TiFlash

    • Support storing the latest data of the storage engine on multiple disks (experimental)

  • TiDB Dashboard

    • Support displaying and sorting by all fields in the SQL Statements page #749
    • Support zooming and panning the topology graph #772
    • Support displaying the disk usage information in the SQL Statements and Slow Queries pages #777
    • Support exporting list data in the SQL Statements and Slow Queries pages #778
    • Support customizing the Prometheus address #808
    • Add a page for cluster statistics #815
    • Add more time-related fields in the Slow Queries details #810

Improvements

  • TiDB

    • Avoid the (index) merge join in a heuristical way when converting equal conditions to other conditions #21146
    • Differentiate the types of user variables #21107
    • Support setting the GOGC variable in the configuration file #20922
    • Make the dumped binary time (Timestamp and Datetime) more compatible with MySQL #21135
    • Provide an error message for statements that use the LOCK IN SHARE MODE syntax #21005
    • Avoid outputting unnecessary warnings or errors when folding constants in shortcut-able expressions #21040
    • Raise an error when preparing the LOAD DATA statement #21199
    • Ignore the attribute of the integer zero-fill size when changing the integer column types #20986
    • Add the executor-related runtime information of DML statements in the result of EXPLAIN ANALYZE #21066
    • Disallow multiple updates on the primary key in a singe SQL statements #21113
    • Add a monitoring metric for the connection idle time #21301
    • Temporarily enable the slow log when the runtime/trace tool is running #20578

  • TiKV

    • Add the tag to trace the source of the split command #8936
    • Support dynamically changing the pessimistic-txn.pipelined configuration #9100
    • Reduce the impact on performance when running Backup & Restore and TiDB Lightning #9098
    • Add monitoring metrics for the ingesting SST errors #9096
    • Prevent the leader from being hibernated when some peers still need to replicate logs #9093
    • Increase the success rate of the pipelined pessimistic locking #9086
    • Change the default value of apply-max-batch-size and store-max-batch-size to 1024 #9020
    • Add the max-background-flushes configuration item #8947
    • Disable force-consistency-checks by default to improve performance #9029
    • Offload the queries on the Region size from pd heartbeat worker to split check worker #9185

  • PD

    • Check the TiKV cluster version when a TiKV stores become Tombstone, which prevents users from enabling incompatible features during the process of downgrade or upgrade #3213
    • Disallow the TiKV store of a lower version to change from Tombstone back to Up #3206

  • TiDB Dashboard

    • Keep expanding when "Expand" is clicked for SQL statements #775
    • Open detail pages in new windows for SQL Statements and Slow Queries #816
    • Improve descriptions for time-related fields in Slow Queries details #817
    • Display detailed error messages #794

  • TiFlash

    • Reduce the latency of replica reads
    • Refine TiFlash's error messages
    • Limit the memory usage of cache data when the data volume is huge
    • Add a monitoring metric for the number of coprocessor tasks being handled

  • Tools

    • Backup & Restore (BR)

      • Disallow the ambiguous --checksum false argument in the command line, which does not correctly disable checksum. Only --checksum=false is accepted. #588
      • Support changing the PD configuration temporarily so that PD can recover the original configuration after BR accidentally exists #596
      • Support analyzing tables after restore #622
      • Retry for the read index not ready and proposal in merging mode errors #626

    • TiCDC

      • Add an alert for enabling TiKV's Hibernate Region feature #1120
      • Reduce memory usage in the schema storage #1127
      • Add the feature of unified sorter, which accelerates replication when the data size of the incremental scan is large (experimental) #1122
      • Support configuring the maximum message size and the maximum message batch in the TiCDC Open Protocol message (only for Kafka sink) #1079

    • Dumpling

      • Retry dumping data on failed chunks #182
      • Support configuring both the -F and -r arguments at the same time #177
      • Exclude system databases in --filter by default #194
      • Support the --transactional-consistency parameter and support rebuilding MySQL connections during retry #199
      • Support using the -c,--compress parameter to specify the compression algorithm used by Dumpling. An empty string means no compression. #202

    • TiDB Lightning

      • Filter out all system schemas by default #459
      • Support setting a default value for the auto-random primary key for the Local-backend or Importer-backend #457
      • Use range properties to make the range split more precise in Local-backend #422
      • Support a human-readable format (such as "2.5 GiB") in tikv-importer.region-split-size, mydumper.read-block-size, mydumper.batch-size, and mydumper.max-region-size #471

    • TiDB Binlog

      • Exit the Drainer process with the non-zero code if the upstream PD is down or if applying DDL or DML statements to the downstream fails #1012

Bug Fixes

  • TiDB

    • Fix the issue of incorrect results when using a prefix index with the OR condition #21287
    • Fix a bug that might cause panic when automatic retry is enabled #21285
    • Fix a bug that occurs when checking partition definition according to column type #21273
    • Fix a bug that the value type of the partition expression is not consistent with the partition column type #21136
    • Fix a bug that the hash-type partition does not check whether the partition name is unique #21257
    • Fix the wrong results returned after inserting a value of the non-INT type into the hash partitioned table #21238
    • Fix the unexpected error when using index join in the INSERT statement in some cases #21249
    • Fix the issue that the BigInt unsigned column value in the CASE WHEN operator is incorrectly converted to the BigInt signed value #21236
    • Fix a bug that index hash join and index merge join do not consider collation #21219
    • Fix a bug that the partitioned table does not consider collation in the CREATE TABLE and SELECT syntax #21181
    • Fix the issue that the query result of slow_query might miss some rows #21211
    • Fix the issue that DELETE might not delete data correctly when the database name is not in a pure lower representation #21206
    • Fix a bug that causes schema change after DML operations #21050
    • Fix the bug that the coalesced column cannot be queried when using join #21021
    • Fix the wrong results of some semi-join queries #21019
    • Fix the issue that the table lock does not take effect on the UPDATE statement #21002
    • Fix the issue of stack overflow that occurs when building the recursive view #21001
    • Fix the unexpected result returned when performing index merge join operations on outer join #20954
    • Fix the issue that sometimes a transaction that has an undetermined result might be treated as failed #20925
    • Fix the issue that EXPLAIN FOR CONNECTION cannot show the last query plan #21315
    • Fix the issue that when Index Merge is used in a transaction with the Read Committed isolation level, the result might be incorrect #21253
    • Fix the auto-ID allocation failure caused by the transaction retry after the write conflict #21079
    • Fix the issue that JSON data cannot be correctly imported to TiDB using LOAD DATA #21074
    • Fix the issue that the default value of newly added Enum-type columns is incorrect #20998
    • Fix the issue that the adddate function inserts invalid characters #21176
    • Fix the issue that the wrong PointGet plan generated in some situations causes wrong results #21244
    • Ignore the conversion of daylight saving time in the ADD_DATE function to be compatible with MySQL #20888
    • Fix a bug that prevents inserting strings with trailing spaces that exceed varchar or char's length constraint #21282
    • Fix a bug that does not converting the integer from [1, 69] to [2001, 2069] or from [70, 99] to [1970, 1999] when comparing int with year #21283
    • Fix the panic caused by the overflowing result of the sum() function when calculating the Double type field #21272
    • Fix a bug that DELETE fails to add lock on the unique key #20705
    • Fix a bug that snapshot reads hits the lock cache #21539
    • Fix an issue of potential memory leak after reading a lot of data in a long-lived transaction #21129
    • Fix the issue that omitting the table alias in a subquery will have a syntax error returned #20367
    • Fix the issue that when the argument of the IN function in a query is the time type, the query might return an incorrect result #21290

  • TiKV

    • Fix the issue that Coprocessor might return wrong results when there are more than 255 columns #9131
    • Fix the issue that Region Merge might cause data loss during network partition #9108
    • Fix the issue that the ANALYZE statement might cause panic when using the latin1 character set #9082
    • Fix the wrong results returned when converting the numeric type to the time type #9031
    • Fix a bug that TiDB Lightning fails to ingest SST files to TiKV with the Importer-backend or Local-backend when Transparent Data Encryption (TDE) is enabled #8995
    • Fix the invalid advertise-status-addr value (0.0.0.0) #9036
    • Fix the issue that an error is returned indicating that a key exists when this key is locked and deleted in a committed transaction #8930
    • Fix the issue that the RocksDB cache mapping error causes data corruption #9029
    • Fix a bug that Follower Read might return stale data after the leader is transferred #9240
    • Fix the issue that stale old values might be read in the pessimistic lock #9282
    • Fix a bug that replica read might get stale data after the leader transfer #9240
    • Fix the issue of TiKV crash that occurs when receiving SIGPROF after profiling #9229

  • PD

    • Fix the issue that the leader roles specified using placement rules do not take effect in some cases #3208
    • Fix the issue that the trace-region-flow value is unexpectedly set to false #3120
    • Fix a bug that the service safepoint with infinite Time To Live (TTL) does not work #3143

  • TiDB Dashboard

    • Fix a display issue of time in the Chinese language #755
    • Fix a bug that the browser compatibility notice does not work #776
    • Fix the issue that the transaction start_ts is incorrectly displayed in some scenarios #793
    • Fix the issue that some SQL texts are incorrectly formatted #805

  • TiFlash

    • Fix the issue that INFORMATION_SCHEMA.CLUSTER_HARDWARE might contain the information of disks that are not in use
    • Fix the issue that the estimate on memory usage of Delta Cache is smaller than the actual usage
    • Fix the memory leak caused by thread information statistics

  • Tools

    • Backup & Restore (BR)

      • Fix the failure caused by special characters in S3 secret access keys #617

    • TiCDC

      • Fix the issue that multiple owners might exist when the owner campaign key is deleted #1104
      • Fix a bug that TiCDC might fail to continue replicating data when a TiKV node crashes or recovers from a crash. This bug only exists in v4.0.8. #1198
      • Fix the issue that the metadata is repeatedly flushed to etcd before a table is initialized #1191
      • Fix an issue of replication interruption caused by early GC or the latency of updating TableInfo when the schema storage caches TiDB tables #1114
      • Fix the issue that the schema storage costs too much memory when DDL operations are frequent #1127
      • Fix the goroutine leak when a changefeed is paused or stopped #1075
      • Increase the maximum retry timeout to 600 seconds in Kafka producer to prevent replication interruption caused by the service or network jitter in the downstream Kafka #1118
      • Fix a bug that the Kafka batch size does not take effect #1112
      • Fix a bug that some tables' row change might be lost when the network between TiCDC and PD has jitter and when there are paused changefeeds being resumed at the same time #1213
      • Fix a bug that the TiCDC process might exit when the network between TiCDC and PD is not stable #1218
      • Use a singleton PD client in TiCDC and fix a bug that TiCDC closes PD client by accident which causes replication block #1217
      • Fix a bug that the TiCDC owner might consume too much memory in the etcd watch client #1224

    • Dumpling

      • Fix the issue that Dumpling might get blocked when its connection to the MySQL database server is closed #190

    • TiDB Lightning

      • Fix the issue that keys are encoded using the wrong field information #437
      • Fix the issue that GC life time TTL does not take effect #448
      • Fix the issue that causes panic when manually stops the running TiDB Lightning in the Local-backend mode #484
Download PDF