Snowflake

Snowflake datasources use the Snowflake native driver for high-performance native data transfer. No ODBC installation is required.

Connection Configuration

URI Format

snowflake://user:password@account/database/schema?warehouse=compute_wh&role=accountadmin

Required Fields

Field	Type	Description
`connection_type`	string	Must be `"snowflake"`
`connection_sub_type`	string	Must be `"database"`
`uri`	string	Snowflake connection string
`server`	string	Snowflake account URL (e.g., `"xy12345.us-east-1.snowflakecomputing.com"`)
`database`	string	Snowflake database name
`username`	string	Snowflake user
`password`	string	Snowflake password
`account`	string	Snowflake account identifier (e.g., `"xy12345.us-east-1"`)
`warehouse`	string	Compute warehouse name
`role`	string	Snowflake role

Optional Fields

Field	Type	Default	Description
`schema`	string		Snowflake schema name

Environment Variables

Variable	Description
`SNOWFLAKE_DEBUGGING`	Enable verbose debug logging for Snowflake connections

Example Connection

{
  "name": "snowflake-market-data",
  "connection_type": "snowflake",
  "connection_sub_type": "database",
  "uri": "snowflake://libuser:s3cureP@ss@xy12345.us-east-1/MARKETDATA/PUBLIC?warehouse=COMPUTE_WH&role=DATA_READER",
  "server": "xy12345.us-east-1.snowflakecomputing.com",
  "database": "MARKETDATA",
  "username": "libuser",
  "password": "s3cureP@ss",
  "account": "xy12345.us-east-1",
  "warehouse": "COMPUTE_WH",
  "role": "DATA_READER",
  "schema": "PUBLIC"
}

The Snowflake native driver uses Snowflake’s native result set format, which means data is returned as data batches directly from the Snowflake service with no row-by-row conversion.

Dataset Configuration (data_args)

Required Fields

Field	Type	Description
`table_options`	list	Tables to query (see below)
`database`	string	Connection reference name
`data_dt_column`	string or list	Column(s) containing the datetime value
`data_dt_format`	bool or string or list	`true` for native datetime auto-detect, or strptime format
`data_dt_timezone`	string	Timezone of the source data
`data_key_column`	string or list	Column(s) used as the symbol/key for query filtering

Optional Fields

Field	Type	Default	Description
`batch_size`	int	`10000`	Number of rows per query batch
`data_dt_nudge`	int	`0`	Microsecond offset applied to timestamps

table_options Entry

Each entry in table_options describes a table to query:

Field	Type	Description
`db`	string	Database name
`table_like`	string	Table name or pattern
`token`	string	Table identifier token
`zone`	string	Timezone, default `"America/New_York"`
`dt_mode`	string	(Optional) Datetime mode
`dt_regex`	string	(Optional) Regex for partitioned tables
`dt_format`	string	(Optional) Datetime format for partitioned tables
`nudge_micros`	int	(Optional) Microsecond nudge

Complete Example

Connection

{
  "name": "snowflake-trades",
  "connection_type": "snowflake",
  "connection_sub_type": "database",
  "uri": "snowflake://libuser:s3cureP@ss@xy12345.us-east-1/TRADES_DB/MARKET?warehouse=COMPUTE_WH&role=accountadmin",
  "server": "xy12345.us-east-1.snowflakecomputing.com",
  "database": "TRADES_DB",
  "username": "libuser",
  "password": "s3cureP@ss",
  "account": "xy12345.us-east-1",
  "warehouse": "COMPUTE_WH",
  "role": "accountadmin",
  "schema": "MARKET"
}

Dataset

{
  "name": "us-equity-trades",
  "connection": "snowflake-trades",
  "data_args": {
    "table_options": [
      {
        "db": "TRADES_DB",
        "table_like": "DAILY_TRADES",
        "token": "daily_trades",
        "zone": "America/New_York"
      }
    ],
    "database": "snowflake-trades",
    "batch_size": 10000,
    "data_dt_column": "TRADE_TIME",
    "data_dt_format": true,
    "data_dt_timezone": "America/New_York",
    "data_dt_nudge": 0,
    "data_key_column": "SYMBOL"
  },
  "schema": [
    { "name": "SYMBOL", "type": "string", "group": "key", "description": "Ticker symbol" },
    { "name": "TRADE_TIME", "type": "string", "group": "time", "description": "Trade timestamp" },
    { "name": "PRICE", "type": "double", "group": "value", "description": "Trade price" },
    { "name": "VOLUME", "type": "int64", "group": "value", "description": "Trade volume" },
    { "name": "EXCHANGE", "type": "string", "group": "value", "description": "Exchange code" }
  ]
}

Snowflake identifiers are uppercase by default. Use uppercase names in your schema definition and data_args fields to match Snowflake’s metadata, unless the objects were created with double-quoted lowercase names.

Warehouse and Role Configuration

The warehouse and role fields control compute resources and access permissions:

Warehouse

The warehouse determines the compute cluster used to execute queries. Choose a warehouse sized appropriately for your query workload:

{
  "warehouse": "COMPUTE_WH"
}

Ensure the specified warehouse is set to auto-resume, or queries will fail when the warehouse is suspended. CloudQuant Data Liberator does not issue ALTER WAREHOUSE ... RESUME commands.

Role

The role determines which database objects are accessible:

{
  "role": "DATA_READER"
}

Create a dedicated read-only role for CloudQuant Data Liberator connections rather than using accountadmin. Grant SELECT on the required databases and schemas to this role.

Debug Logging

Enable verbose debug logging for Snowflake connections to diagnose connectivity or query issues:

export SNOWFLAKE_DEBUGGING=1

This produces detailed logs including connection negotiation, query execution timing, and data batch transfer metrics.

Driver and Dependencies

Component	Details
Driver	`Snowflake native driver`
Protocol	Snowflake REST API / streaming protocol
Data conversion	Native (high-performance)
ODBC required	No
Default port	443 (HTTPS)

The Snowflake native driver is bundled with CloudQuant Data Liberator. No additional driver installation is needed on the host system. All communication uses HTTPS on port 443.

Troubleshooting

Account identifier format

The account field must include the region if your account is not in the default AWS us-west-2 region:

# US East (Ohio)
xy12345.us-east-1

# EU (Frankfurt)
xy12345.eu-central-1

# Azure (East US 2)
xy12345.east-us-2.azure

Authentication failed (390100)

Verify the username, password, and account identifier. Snowflake passwords are case-sensitive. If multi-factor authentication (MFA) is enabled for the user, you may need to use a service account without MFA.

Warehouse is suspended

If queries fail with a warehouse suspension error, ensure the warehouse is configured with AUTO_RESUME = TRUE:

ALTER WAREHOUSE COMPUTE_WH SET AUTO_RESUME = TRUE;

Role does not have access

Verify the role has the required privileges:

GRANT USAGE ON DATABASE TRADES_DB TO ROLE DATA_READER;
GRANT USAGE ON SCHEMA TRADES_DB.MARKET TO ROLE DATA_READER;
GRANT SELECT ON ALL TABLES IN SCHEMA TRADES_DB.MARKET TO ROLE DATA_READER;

Network connectivity

Snowflake connections require HTTPS (port 443) access to *.snowflakecomputing.com. Ensure firewalls and proxy servers allow this traffic.

Slow queries

Increase batch_size for large result sets (up to 100000).
Ensure Snowflake clustering keys align with your datetime and key columns.
Use an appropriately sized warehouse (e.g., MEDIUM or LARGE) for large datasets.
Check that the warehouse is not queued behind other workloads using Snowflake’s query history.

Connections

​Snowflake

​Connection Configuration

​URI Format

​Required Fields

​Optional Fields

​Environment Variables

​Example Connection

​Dataset Configuration (data_args)

​Required Fields

​Optional Fields

​table_options Entry

​Complete Example

​Connection

​Dataset

​Warehouse and Role Configuration

​Warehouse

​Role

​Debug Logging

​Driver and Dependencies

​Troubleshooting

​Account identifier format

​Authentication failed (390100)

​Warehouse is suspended

​Role does not have access

​Network connectivity

​Slow queries

Snowflake

Connection Configuration

URI Format

Required Fields

Optional Fields

Environment Variables

Example Connection

Dataset Configuration (data_args)

Required Fields

Optional Fields

table_options Entry

Complete Example

Connection

Dataset

Warehouse and Role Configuration

Warehouse

Role

Debug Logging

Driver and Dependencies

Troubleshooting

Account identifier format

Authentication failed (390100)

Warehouse is suspended

Role does not have access

Network connectivity

Slow queries