Azure Blob Storage

Azure Blob Storage datasources allow CloudQuant Data Liberator to read CSV, TSV, and other delimited files from Azure Blob containers. CloudQuant Data Liberator mounts the container via FUSE.

See Supported Data Formats for every file extension Liberator can ingest on Azure Blob, including formats added in 2.1 and 2.2.

Connection configuration

Required fields

Field	Type	Description
`connection_type`	string	Must be `"azure"`
`account`	string	Azure storage account name
`key`	string	Storage account access key
`container`	string	Blob container name

Optional fields

Field	Type	Default	Description
`endpoint`	string		Blob endpoint URL (override for sovereign clouds or Azurite)
`prefix`	string	`""`	Blob prefix (virtual directory) within the container
`mount_point`	string		Local mount path for FUSE-based access
`config_name`	string		Internal configuration identifier
`object_key`	string		Object key pattern for blob selection

The endpoint field is only needed for non-standard Azure environments such as Azure Government, Azure China, or local development with Azurite. For standard Azure public cloud, CloudQuant Data Liberator derives the endpoint from the account name.

Example connection

{
  "name": "azure-market-data",
  "connection_type": "azure",
  "account": "mystorageaccount",
  "key": "base64encodedstorageaccountkey==",
  "container": "market-data",
  "prefix": "equities/daily/"
}

Storage account keys grant full access to the entire storage account. Rotate keys regularly and use environment variables or a secrets manager to inject them at deployment time.

Dataset configuration (data_args)

The data_args fields are identical to all file-based sources. See Local File for the full reference. The file_pattern is evaluated relative to the prefix configured on the connection.

Required data_args

Field	Type	Description
`file_pattern`	string	Glob pattern relative to the prefix, e.g., `"*.csv"`
`data_dt_column`	string or list	Column(s) containing the datetime value
`data_dt_format`	string or list	strptime format or special values (`"muts"`, `"uts"`, `"datetime"`)
`data_key_column`	string or list	Symbol/key column(s)

Optional data_args

Field	Type	Default	Description
`sep_override`	string	`","`	Delimiter character
`encoding`	string	`"utf-8"`	File encoding
`data_dt_timezone`	string	`"UTC"`	Source data timezone
`fname_dt_regex`	string		Regex to extract date from filename
`fname_dt_format`	string		strptime format for filename date
`fname_dt_timezone`	string		Timezone of filename date
`fname_dt_nudge`	int	`0`	Microsecond offset for filename date
`fname_dt_approx_seconds`	int		Approximate seconds per file
`arrow_sort`	list	`["symbol", "muts"]`	Sort order
`arrow_timestamp`	bool	`true`	Generate human-readable timestamp column

Complete example

Connection

{
  "name": "azure-eod-prices",
  "connection_type": "azure",
  "account": "caboretlstorage",
  "key": "xYzAbCdEfGhIjKlMnOpQrStUvWxYz0123456789==",
  "container": "eod-data",
  "prefix": "prices/us-equities/"
}

Dataset

{
  "name": "us-equity-eod",
  "connection": "azure-eod-prices",
  "data_args": {
    "file_pattern": "eod_*.csv",
    "sep_override": ",",
    "encoding": "utf-8",
    "data_dt_column": "date",
    "data_dt_format": "%Y-%m-%d",
    "data_dt_timezone": "America/New_York",
    "data_key_column": "ticker",
    "fname_dt_regex": "eod_(\\d{8})\\.csv",
    "fname_dt_format": "%Y%m%d",
    "fname_dt_timezone": "America/New_York",
    "fname_dt_approx_seconds": 86400,
    "arrow_sort": ["symbol", "muts"],
    "arrow_timestamp": true
  },
  "schema": [
    { "name": "ticker", "type": "string", "group": "key", "description": "Ticker symbol" },
    { "name": "date", "type": "string", "group": "time", "description": "Trade date" },
    { "name": "open", "type": "double", "group": "value", "description": "Open price" },
    { "name": "high", "type": "double", "group": "value", "description": "High price" },
    { "name": "low", "type": "double", "group": "value", "description": "Low price" },
    { "name": "close", "type": "double", "group": "value", "description": "Close price" },
    { "name": "volume", "type": "int64", "group": "value", "description": "Daily volume" },
    { "name": "adj_close", "type": "double", "group": "value", "description": "Adjusted close price" }
  ]
}

Azure setup requirements

Ensure the following before configuring your Azure Blob connection:

Storage account exists and is accessible from the CloudQuant Data Liberator host
Container is created within the storage account
Access key is available (found under Storage Account > Access Keys in the Azure Portal)
Network access is permitted — if the storage account uses firewall rules, add the CloudQuant Data Liberator host’s IP to the allowed list

For initial testing, use Azure Storage Explorer to verify your files are accessible before configuring the CloudQuant Data Liberator connection. This eliminates credential and network issues early.

See the Azure Blob Storage integration guide for additional setup details.

​Azure Blob Storage

​Connection configuration

​Required fields

​Optional fields

​Example connection

​Dataset configuration (data_args)

​Required data_args

​Optional data_args

​Complete example

​Connection

​Dataset

​Azure setup requirements

Azure Blob Storage

Connection configuration

Required fields

Optional fields

Example connection

Dataset configuration (data_args)

Required data_args

Optional data_args

Complete example

Connection

Dataset

Azure setup requirements