Root directory structure

QuestDB creates the following file structure in its root_directory:

questdb
├── conf
├── db
├── log
├── public
└── snapshot (optional)

By default, QuestDB's root directory will be the following:

Linux

$HOME/.questdb

macOS (Homebrew)

Path on Macs with Apple Silicon (M1 or M2) chip:

/opt/homebrew/var/questdb

Path on Macs with Intel chip:

/usr/local/var/questdb

Windows

C:\Windows\System32\qdbroot

conf directory

Contains configuration files for QuestDB:

├── conf
│   ├── date.formats
│   ├── mime.types
│   └── server.conf

file	description
date.formats	A list of date formats in plain text.
mime.types	Mapping file used by the HTTP server to map file extension to response type when an user downloads a file.
server.conf	Server configuration file. Find out more in the server configuration section.

db directory

This directory contains all the files related to database tables. It is organised as follows:

• Each table has its own table_directory under root_directory/db/table_name
• Within a table_directory, each partition has its own partition_directory.
• Within each partition directory, each column has its own column_file, for example mycolumn.d
• If a given column has an index, then there will also be an index_file, for example mycolumn.k

Table and Column Versioning

QuestDB uses a copy-on-write mechanism for data updates, which means that when you update existing data, new versions of column files are created rather than modifying existing ones. This ensures data consistency and allows readers to continue accessing the previous version while updates are in progress.

For more details on how versioning works, see the Updating data documentation.

The table also stores metadata in _meta files:

├── db
│   ├── Table
│   │   │
│   │   ├── Partition 1
│   │   │   ├── _archive
│   │   │   ├── column1.d
│   │   │   ├── column2.d
│   │   │   ├── column2.k
│   │   │   └── ...
│   │   ├── Partition 2
│   │   │   ├── _archive
│   │   │   ├── column1.d
│   │   │   ├── column2.d
│   │   │   ├── column2.k
│   │   │   └── ...
│   │   │
│   │   ├── _meta
│   │   ├── _txn
│   │   └── _cv
│   └──  table_1.lock

Column Versioning in Practice

When data is updated, you may see multiple versions of column files in the directory structure. For example, after updating column1, you might see:

├── Partition 1
│   ├── column1.d        # Original version
│   ├── column1.d.1      # New version after update
│   ├── column2.d        # Unchanged column
│   └── ...

The _cv (column version) metadata file tracks which version of each column is currently active. Older versions are automatically cleaned up by the vacuum process.

Metadata files

Each table directory contains several metadata files:

file	description
_meta	Table metadata including schema information
_txn	Transaction metadata for consistency
_cv	Column version metadata - tracks which version of each column is currently active

If the table is not partitioned, data is stored in a directory called default:

├── db
│   ├── Table
│   │   │  
│   │   ├── default
│   │   │   ├── _archive
│   │   │   ├── column1.d
│   │   │   ├── column2.d
│   │   │   ├── column2.k
│   │   │   └── ...
│   │   ├── _meta
│   │   └── _txn
│   └──  table_1.lock

For a WAL table, the table structure contains one or more wal folders and a seq folder representing the Sequencer:

├── db
│   ├── Table
│   │   │  
│   │   ├── Partition 1
│   │   │   ├── _archive
│   │   │   ├── column1.d
│   │   │   ├── column2.d
│   │   │   ├── column2.k
│   │   │   └── ...
│   │   ├── Partition 2
│   │   │   ├── _archive
│   │   │   ├── column1.d
│   │   │   ├── column2.d
│   │   │   ├── column2.k
│   │   │   └── ...
│   │   ├── txn_seq
│   │   │   ├── _meta
│   │   │   ├── _txnlog
│   │   │   └── _wal_index.d
│   │   ├── wal1
│   │   │   └── 0
│   │   │       ├── _meta
│   │   │       ├── _event
│   │   │       ├── column1.d
│   │   │       ├── column2.d
│   │   │       └── ...
|   |   | 
│   │   ├── wal2
│   │   │   └── 0
│   │   │   │   ├── _meta
│   │   │   │   ├── _event
│   │   │   │   ├── column1.d
│   │   │   │   ├── column2.d
│   │   │   │   └── ...
│   │   │   └── 1
│   │   │       ├── _meta
│   │   │       ├── _event
│   │   │       ├── column1.d
│   │   │       ├── column2.d
│   │   │       └── ...
│   │   │ 
│   │   ├── _meta
│   │   ├── _txn
│   │   └── _cv
│   |

caution

As tempting as it may be to delete partitions by manually removing the directories from the file system, we really discourage this. The partitions are organised with metadata and deleting them directly could corrupt the table. We recommend you use ALTER TABLE DROP PARTITION for this effect.

log directory

Contains the log files for QuestDB:

├── log
│   ├── stdout-2020-04-15T11-59-59.txt
│   └── stdout-2020-04-12T13-31-22.txt

Log files look like this:

2020-04-15T16:42:32.879970Z I i.q.c.TableReader new transaction [txn=2, transientRowCount=1, fixedRowCount=1, maxTimestamp=1585755801000000, attempts=0]
2020-04-15T16:42:32.880051Z I i.q.g.FunctionParser call to_timestamp('2020-05-01:15:43:21','yyyy-MM-dd:HH:mm:ss') -> to_timestamp(Ss)
2020-04-15T16:42:32.880657Z I i.q.c.p.WriterPool >> [table=`table_1`, thread=12]
2020-04-15T16:42:32.881330Z I i.q.c.AppendMemory truncated and closed [fd=32]
2020-04-15T16:42:32.881448Z I i.q.c.AppendMemory open /usr/local/var/questdb/db/table_1/2020-05/timestamp.d [fd=32, pageSize=16777216]
2020-04-15T16:42:32.881708Z I i.q.c.AppendMemory truncated and closed [fd=33]
2020-04-15T16:42:32.881830Z I i.q.c.AppendMemory open /usr/local/var/questdb/db/table_1/2020-05/temperature.d [fd=33, pageSize=16777216]
2020-04-15T16:42:32.882092Z I i.q.c.AppendMemory truncated and closed [fd=34]
2020-04-15T16:42:32.882210Z I i.q.c.AppendMemory open /usr/local/var/questdb/db/table_1/2020-05/humidity.d [fd=34, pageSize=16777216]
2020-04-15T16:42:32.882248Z I i.q.c.TableWriter switched partition to '/usr/local/var/questdb/db/table_1/2020-05'
2020-04-15T16:42:32.882571Z I i.q.c.p.WriterPool << [table=`table_1`, thread=12]
2020-04-15T16:44:33.245144Z I i.q.c.AppendMemory truncated and closed [fd=32]
2020-04-15T16:44:33.245418Z I i.q.c.AppendMemory truncated and closed [fd=33]
2020-04-15T16:44:33.245712Z I i.q.c.AppendMemory truncated and closed [fd=34]
2020-04-15T16:44:33.246096Z I i.q.c.ReadWriteMemory truncated and closed [fd=30]
2020-04-15T16:44:33.246217Z I i.q.c.ReadOnlyMemory closed [fd=31]
2020-04-15T16:44:33.246461Z I i.q.c.TableWriter closed 'table_1'
2020-04-15T16:44:33.246492Z I i.q.c.p.WriterPool closed [table=`table_1`, reason=IDLE, by=12]
2020-04-15T16:44:33.247184Z I i.q.c.OnePageMemory closed [fd=28]
2020-04-15T16:44:33.247239Z I i.q.c.ReadOnlyMemory closed [fd=27]
2020-04-15T16:44:33.247267Z I i.q.c.TableReader closed 'table_1'
2020-04-15T16:44:33.247287Z I i.q.c.p.ReaderPool closed 'table_1' [at=0:0, reason=IDLE]
2020-04-15T16:44:39.763406Z I http-server disconnected [ip=127.0.0.1, fd=24]
2020-04-15T16:44:39.763729Z I i.q.c.h.HttpServer pushed

public directory

Contains the web files for the Web Console:

└── public
    ├── assets
    │   ├── console-configuration.json
    │   └── favicon.webp
    ├── index.html
    ├── qdb.js
    ├── qdb.css
    └── ...

snapshot directory

Created when a filesystem (disk) snapshot is collected. Contains table metadata file copies.

tmp directory

Created when a COPY SQL command is run for a partitioned table and no value is set for the cairo.sql.copy.work.root configuration setting. Contains temporary import files like indexes or temporary partitions.