TokuDB files and file types¶
Starting with Percona Server for MySQL 8.0.28-19 (2022-05-12), the TokuDB storage engine is no longer supported. For more information, see the TokuDB Introduction and TokuDB version changes.
The TokuDB file set consists of many different files that all serve various purposes.
If you have any TokuDB data, your data directory should look similar to this:
root@server:/var/lib/mysql# ls -lah
The output could be:
...
-rw-rw---- 1 mysql mysql 76M Oct 13 18:45 ibdata1
...
-rw-rw---- 1 mysql mysql 16K Oct 13 15:52 tokudb.directory
-rw-rw---- 1 mysql mysql 16K Oct 13 15:52 tokudb.environment
-rw------- 1 mysql mysql 0 Oct 13 15:52 __tokudb_lock_dont_delete_me_data
-rw------- 1 mysql mysql 0 Oct 13 15:52 __tokudb_lock_dont_delete_me_environment
-rw------- 1 mysql mysql 0 Oct 13 15:52 __tokudb_lock_dont_delete_me_logs
-rw------- 1 mysql mysql 0 Oct 13 15:52 __tokudb_lock_dont_delete_me_recovery
-rw------- 1 mysql mysql 0 Oct 13 15:52 __tokudb_lock_dont_delete_me_temp
-rw-rw---- 1 mysql mysql 16K Oct 13 15:52 tokudb.rollback
...
This document lists the different types of TokuDB and Percona Fractal Tree files, explains their purpose, shows their location and how to move them around.
tokudb.environment¶
This file is the root of the Percona FT file set and contains various bits of metadata about the system, such as creation times, current file format versions, etc.
Percona FT will create/expect this file in the directory specified by the MySQL datadir.
tokudb.rollback¶
Every transaction within Percona FT maintains its own transaction rollback log. These logs are stored together within a single Percona FT dictionary file and take up space within the Percona FT cachetable (just like any other Percona FT dictionary).
The transaction rollback logs will undo any changes made by a transaction
if the transaction is explicitly rolled back, or rolled back via recovery as a
result of an uncommitted transaction when a crash occurs.
Percona FT will create/expect this file in the directory specified by the MySQL datadir.
tokudb.directory¶
Percona FT maintains a mapping of a dictionary name (example:
sbtest.sbtest1.main) to an internal file name (example:
_sbtest_sbtest1_main_xx_x_xx.tokudb). This mapping is stored within this
single Percona FT dictionary file and takes up space within the Percona FT
cachetable just like any other Percona FT dictionary.
Percona FT will create/expect this file in the directory specified by the MySQL datadir.
Dictionary files¶
TokuDB dictionary (data) files store actual user data. For each MySQL table there will be:
-
One
statusdictionary that contains metadata about the table. -
One
maindictionary that stores the full primary key (an imaginary key is used if one was not explicitly specified) and full row data. -
One
keydictionary for each additional key/index on the table.
These are typically named:
_<database>_<table>_<key>_<internal_txn_id>.tokudb
Percona FT creates/expects these files in the directory specified by
tokudb_data_dir if set, otherwise the MySQL datadir is used.
Recovery log files¶
The Percona FT recovery log records every operation that modifies a Percona FT dictionary. Periodically, the system will take a snapshot of the system called a checkpoint. This checkpoint ensures that the modifications recorded within the Percona FT recovery logs have been applied to the appropriate dictionary files up to a known point in time and synced to disk.
These files have a rolling naming convention, but use:
log<log_file_number>.tokulog<log_file_format_version>.
Percona FT creates/expects these files in the directory specified by tokudb_log_dir if set, otherwise the MySQL datadir is used.
Percona FT does not track what log files should or shouldn’t be present. Upon startup, it discovers the logs in the log directory, and replays them in order. If the wrong logs are present, the recovery aborts and possibly damages the dictionaries.
Temporary files¶
Percona FT might need to create some temporary files in order to perform some operations. When the bulk loader is active, these temporary files might grow to be quite large.
As different operations start and finish, the files will come and go.
There are no temporary files left behind upon a clean shutdown,
Percona FT creates/expects these files in the directory specified by tokudb_tmp_dir if set. If not, the tokudb_data_dir is used if set, otherwise the MySQL datadir is used.
Lock files¶
Percona FT uses lock files to prevent multiple processes from accessing and writing to the files in the assorted Percona FT functionality areas. Each lock file will be in the same directory as the file(s) that it is protecting.
These empty files are only used as semaphores across processes. They are safe to delete/ignore as long as no server instances are currently running and using the data set.
__tokudb_lock_dont_delete_me_environment
__tokudb_lock_dont_delete_me_recovery
__tokudb_lock_dont_delete_me_logs
__tokudb_lock_dont_delete_me_data
__tokudb_lock_dont_delete_me_temp
Percona FT validates its data set. If a file goes missing or not found, or seems to contain some nonsensical data, it will assert, abort or fail to start. It does this not to annoy you, but to try to protect you from doing any further damage to your data.
Get expert help¶
If you need assistance, visit the community forum for comprehensive and free database knowledge, or contact our Percona Database Experts for professional support and services.