Azure Blob Storage
Azure Blob Storage datasources allow CloudQuant Data Liberator to read CSV, TSV, and other delimited files from Azure Blob containers. CloudQuant Data Liberator mounts the container via FUSE.
Connection Configuration
Required Fields
| Field | Type | Description |
|---|
connection_type | string | Must be "azure" |
account | string | Azure storage account name |
key | string | Storage account access key |
container | string | Blob container name |
Optional Fields
| Field | Type | Default | Description |
|---|
endpoint | string | | Blob endpoint URL (override for sovereign clouds or Azurite) |
prefix | string | "" | Blob prefix (virtual directory) within the container |
mount_point | string | | Local mount path for FUSE-based access |
config_name | string | | Internal configuration identifier |
object_key | string | | Object key pattern for blob selection |
The endpoint field is only needed for non-standard Azure environments such as Azure Government, Azure China, or local development with Azurite. For standard Azure public cloud, CloudQuant Data Liberator derives the endpoint from the account name.
Example Connection
{
"name": "azure-market-data",
"connection_type": "azure",
"account": "mystorageaccount",
"key": "base64encodedstorageaccountkey==",
"container": "market-data",
"prefix": "equities/daily/"
}
Storage account keys grant full access to the entire storage account. Rotate keys regularly and use environment variables or a secrets manager to inject them at deployment time.
Dataset Configuration (data_args)
The data_args fields are identical to all file-based sources. See Local File for the full reference. The file_pattern is evaluated relative to the prefix configured on the connection.
Required data_args
| Field | Type | Description |
|---|
file_pattern | string | Glob pattern relative to the prefix, e.g., "*.csv" |
data_dt_column | string or list | Column(s) containing the datetime value |
data_dt_format | string or list | strptime format or special values ("muts", "uts", "datetime") |
data_key_column | string or list | Symbol/key column(s) |
Optional data_args
| Field | Type | Default | Description |
|---|
sep_override | string | "," | Delimiter character |
encoding | string | "utf-8" | File encoding |
data_dt_timezone | string | "UTC" | Source data timezone |
fname_dt_regex | string | | Regex to extract date from filename |
fname_dt_format | string | | strptime format for filename date |
fname_dt_timezone | string | | Timezone of filename date |
fname_dt_nudge | int | 0 | Microsecond offset for filename date |
fname_dt_approx_seconds | int | | Approximate seconds per file |
arrow_sort | list | ["symbol", "muts"] | Sort order |
arrow_timestamp | bool | true | Generate human-readable timestamp column |
Complete Example
Connection
{
"name": "azure-eod-prices",
"connection_type": "azure",
"account": "caboretlstorage",
"key": "xYzAbCdEfGhIjKlMnOpQrStUvWxYz0123456789==",
"container": "eod-data",
"prefix": "prices/us-equities/"
}
Dataset
{
"name": "us-equity-eod",
"connection": "azure-eod-prices",
"data_args": {
"file_pattern": "eod_*.csv",
"sep_override": ",",
"encoding": "utf-8",
"data_dt_column": "date",
"data_dt_format": "%Y-%m-%d",
"data_dt_timezone": "America/New_York",
"data_key_column": "ticker",
"fname_dt_regex": "eod_(\\d{8})\\.csv",
"fname_dt_format": "%Y%m%d",
"fname_dt_timezone": "America/New_York",
"fname_dt_approx_seconds": 86400,
"arrow_sort": ["symbol", "muts"],
"arrow_timestamp": true
},
"schema": [
{ "name": "ticker", "type": "string", "group": "key", "description": "Ticker symbol" },
{ "name": "date", "type": "string", "group": "time", "description": "Trade date" },
{ "name": "open", "type": "double", "group": "value", "description": "Open price" },
{ "name": "high", "type": "double", "group": "value", "description": "High price" },
{ "name": "low", "type": "double", "group": "value", "description": "Low price" },
{ "name": "close", "type": "double", "group": "value", "description": "Close price" },
{ "name": "volume", "type": "int64", "group": "value", "description": "Daily volume" },
{ "name": "adj_close", "type": "double", "group": "value", "description": "Adjusted close price" }
]
}
Azure Setup Requirements
Ensure the following before configuring your Azure Blob connection:
- Storage account exists and is accessible from the CloudQuant Data Liberator host
- Container is created within the storage account
- Access key is available (found under Storage Account > Access Keys in the Azure Portal)
- Network access is permitted — if the storage account uses firewall rules, add the CloudQuant Data Liberator host’s IP to the allowed list
For initial testing, use Azure Storage Explorer to verify your files are accessible before configuring the CloudQuant Data Liberator connection. This eliminates credential and network issues early.
