​

Supported data formats

CloudQuant Data Liberator ingests time-series and tabular data through connections (where data lives) and datasets (how files or tables are interpreted). The tables below list every file extension and data source category the platform understands. Formats are selected automatically from your file_pattern, file extension, or connection type when you onboard through the Liberator UI. You rarely need to set anything manually.

​

Supported in version 2.0

These formats were available in the Liberator 2.0 release and remain supported on current versions.

​

Delimited text files

Set sep_override in data_args to force comma, tab, pipe (\|), or semicolon when auto-detection is not sufficient.

​

Columnar and binary files

On S3, Parquet datasets can use passthrough mode (direct read without intermediate caching). See the S3 Parquet example.

​

Structured office and scientific files

​

Archives

​

Database sources

​

API-backed datasets

REST endpoints that return JSON tabular payloads can be onboarded as API connections. The platform normalizes responses into the same query surface as file- and database-backed datasets.

​

Schema column types

Regardless of source format, dataset schemas use these column types: string, int64, uint64, double, float, bool, date32, date64, time64 See Datasource configuration overview for column groups (key, time, value, meta).

​

Formats added after version 2.0

The following ingest formats were added in subsequent releases. They are available on current CloudQuant-managed environments at those versions and later.

​

Version 2.1 — PSV (pipe-separated values)

PSV files use the same onboarding flow as CSV: header row, per-column type inference, and configurable null sentinel. You can also ingest pipe-delimited .csv or .txt files by setting sep_override to "\|" without renaming the file.

​

Version 2.2 — PCAP / PCAPng (FIX tick capture)

Liberator extracts FIX-protocol messages from TCP payloads in packet captures and exposes them through the standard query API. Typical columns include FIX tags such as 35 (MsgType), 49 (SenderCompID), 52 (SendingTime), and 55 (Symbol), plus _pcap_ts_ns for the capture timestamp. Use PCAP datasets when you capture exchange feeds at the wire level and want the same query model as historical bar or trade datasets.

​

Added in version 2.3

No new file or data ingest formats are part of the 2.3 release at this time. See What’s New in Liberator 2.2 for PCAP/PCAPng and What’s New in Liberator 2.1 for PSV.

​

Where each format applies

All file-based formats in the tables above can be stored on any file-backed connection type: Database and API formats map to their respective connection guides:

​

Choosing a format

​

Related