Symbols and Key Fields for Queries
The symbol parameter is our KEY for a dataset, this is because our first datasets are all related to US Equity Symbols. As time marched on, and our range of datasets provided increased (to include such varying data types as shipping, weather, and market research with keys as diverse as zip codes, geo-locations, and age groups to name a few), we decided it would cause less confusion for our existing customers if we continued with the symbol parameter as our KEY when querying a dataset. You will get used to it, I promise!
DEFAULT VALUE: None (effectively returns all Symbols/Keys including blanks/Nan’s)
REMINDER: When setting your query parameters, consider the amount of data you may be requesting and start small if you can until you get a good idea of how much data is returned for each symbol/date range combination.
How do I know how many unique Symbols/Keys are in a particular dataset in a particular date range?
Whilst there are more sophisticated methods available on some Liberators, generally the simplest method is to pull the data and count. For example, lets go back to the daily_bars dataset and pull 2 days of data but all symbols.
How to query all data for all Symbols/Keys in a dataset
Let do the same query as above but this time we will pull just two days and all Symbols/Keys.
There are two methods for getting all Symbols/Keys, either set the parameter symbols to None or drop the parameter completely and it will default to None ie…
df = liberator.get_dataframe(liberator.query( name = 'daily_bars', as_of ='2024-07-24', back_to = '2024-07-22', symbols = None))
df
This produces a dataframe with over 20k rows. To identify how many unique symbols are contained in the dataset you can run this python command…
len(df.symbol.unique())
10524
So 10524 unique symbols over the two days.
To get a list of the symbols and store that list into a variable try this…
allsym = list(df.symbol.unique())
To display this list in, say, jupyter notebooks/labs try this..
print(list(df.symbol.unique()))
CAUTION - this can be quite a long list!