MySQL
MySQL datasources use the ODBC driver and the MariaDB ODBC connector. The connection string is built internally from the provided fields.
Connection Configuration
mysql://user:password@host:3306/database
The URI is used as a reference identifier. CloudQuant Data Liberator builds the actual ODBC connection string internally from the individual connection fields (server, database, username, password).
Required Fields
| Field | Type | Description |
|---|
connection_type | string | Must be "mysql" |
connection_sub_type | string | Must be "database" |
uri | string | MySQL connection string |
server | string | Database host address |
database | string | Database name |
username | string | Database user |
password | string | Database password |
Optional Fields
| Field | Type | Default | Description |
|---|
ssl_disabled | string | "False" | Set to "True" to disable SSL. Values: "True" / "False" |
Example Connection
{
"name": "mysql-market-data",
"connection_type": "mysql",
"connection_sub_type": "database",
"uri": "mysql://libuser:s3cureP@ss@db.example.com:3306/marketdata",
"server": "db.example.com",
"database": "marketdata",
"username": "libuser",
"password": "s3cureP@ss",
"ssl_disabled": "False"
}
Dataset Configuration (data_args)
Required Fields
| Field | Type | Description |
|---|
table_options | list | Tables to query (see below) |
database | string | Connection reference name |
data_dt_column | string or list | Column(s) containing the datetime value |
data_dt_format | bool or string or list | true for native datetime auto-detect, or strptime format |
data_dt_timezone | string | Timezone of the source data |
data_key_column | string or list | Column(s) used as the symbol/key for query filtering |
Optional Fields
| Field | Type | Default | Description |
|---|
batch_size | int | 10000 | Number of rows per query batch |
data_dt_nudge | int | 0 | Microsecond offset applied to timestamps |
table_options Entry
Each entry in table_options describes a table to query:
| Field | Type | Description |
|---|
db | string | Database name |
table_like | string | Table name or pattern |
token | string | Table identifier token |
zone | string | Timezone, default "America/New_York" |
dt_mode | string | (Optional) Datetime mode |
dt_regex | string | (Optional) Regex for partitioned tables |
dt_format | string | (Optional) Datetime format for partitioned tables |
nudge_micros | int | (Optional) Microsecond nudge |
Complete Example
Connection
{
"name": "mysql-trades",
"connection_type": "mysql",
"connection_sub_type": "database",
"uri": "mysql://libuser:s3cureP@ss@db.example.com:3306/trades_db",
"server": "db.example.com",
"database": "trades_db",
"username": "libuser",
"password": "s3cureP@ss",
"ssl_disabled": "False"
}
Dataset
{
"name": "us-equity-trades",
"connection": "mysql-trades",
"data_args": {
"table_options": [
{
"db": "trades_db",
"table_like": "daily_trades",
"token": "daily_trades",
"zone": "America/New_York"
}
],
"database": "mysql-trades",
"batch_size": 10000,
"data_dt_column": "trade_time",
"data_dt_format": true,
"data_dt_timezone": "America/New_York",
"data_dt_nudge": 0,
"data_key_column": "symbol"
},
"schema": [
{ "name": "symbol", "type": "string", "group": "key", "description": "Ticker symbol" },
{ "name": "trade_time", "type": "string", "group": "time", "description": "Trade timestamp" },
{ "name": "price", "type": "double", "group": "value", "description": "Trade price" },
{ "name": "volume", "type": "int64", "group": "value", "description": "Trade volume" },
{ "name": "exchange", "type": "string", "group": "value", "description": "Exchange code" }
]
}
Set data_dt_format to true when using native MySQL DATETIME or TIMESTAMP columns. CloudQuant Data Liberator will auto-detect the datetime type and convert directly without string parsing.
SSL Configuration
By default, SSL is enabled. To disable SSL (e.g., for local development):
{
"ssl_disabled": "True"
}
Disabling SSL in production is not recommended. Always use SSL when connecting over untrusted networks. The MariaDB ODBC driver supports TLS 1.2+ by default.
Driver and Dependencies
| Component | Details |
|---|
| Driver | ODBC driver with MariaDB ODBC 3.2+ |
| Protocol | ODBC |
| Arrow conversion | High-performance ODBC driver |
| ODBC required | Yes (MariaDB ODBC connector) |
| Default port | 3306 |
The MariaDB ODBC 3.2+ driver must be installed on the CloudQuant Data Liberator host. This driver is compatible with both MySQL and MariaDB servers. Verify installation with odbcinst -q -d.
Troubleshooting
ODBC driver not found
If you see errors about missing ODBC drivers, verify the MariaDB ODBC driver is installed and registered:
# List installed ODBC drivers
odbcinst -q -d
# Expected output should include:
# [MariaDB ODBC 3.2 Driver]
# Debian/Ubuntu
apt-get install libmariadb3 odbc-mariadb
# RHEL/CentOS
yum install MariaDB-connector-odbc
Connection refused
Verify the MySQL server is accepting connections on the specified host and port. Check that the bind-address in the MySQL configuration allows remote connections.
mysql -h db.example.com -P 3306 -u libuser -p marketdata
Authentication failed
Ensure the user has been granted access from the CloudQuant Data Liberator host’s IP address:
GRANT SELECT ON trades_db.* TO 'libuser'@'liberator-host' IDENTIFIED BY 's3cureP@ss';
FLUSH PRIVILEGES;
SSL handshake errors
If SSL connections fail, check that the MySQL server’s SSL certificate is valid and trusted. Set ssl_disabled to "True" temporarily to confirm the issue is SSL-related.
Slow queries
- Increase
batch_size for large result sets (up to 100000).
- Ensure the datetime and key columns are indexed in MySQL.
- For partitioned tables, ensure
dt_regex is correctly scoping queries to relevant partitions.
