Accessing Point-in-time data in Mariner
The Liberator library allows simplified access to alternative data sets of all different sizes and shapes.
As a user you simply supply the symbol(s) and dataset you are interested in and the as_of/back_to dates.
Liberator then does all the heavy lifting in the background and presents you with the results.
🔥🎵 One should only call liberator from Strategy.on_strategy_start()
Otherwise, you will likely create thousands of queries simultaneously accessing the Liberator data warehouse. Liberator has "denial of service" protections that will not allow this query pattern.
Example
import liberator as b24
df = b24.get_dataframe(b24.query(symbols = ['AAPL'], as_of = '2020-01-25', back_to = '2020-01-20', name = 'DXOpen SF'))
Query
b24.query(symbols=None, as_of=None, back_to=None, stats=None, name=None)
| symbols | String of list of strings to query for, if None all symbols are queried |
| as_of | Muts or date string in form "YYYY-mm-dd HH:MM:SS.ffffff". The date at which data should be assessed (what did we know at this time?) |
| back_to | Muts or date string in form "YYYY-mm-dd HH:MM:SS.ffffff". The furthest back to look for data at |
| stats | True/False see below |
| name | The name of the dataset being queried |
Query Returns
stats = True : query will return statistics about what the results of the query look like. Useful for large datasets to make sure you don't try to read a massive result into a dataframe.
stats = False : query will return a 'generator type', which can be either converted to a dataframe or iterated through (for very large datasets).
Convert query results to a Pandas Dataframe
res = b24.query(symbols = ['AAPL'], as_of = '2020-01-25', back_to = '2020-01-20', name = 'DXOpen SF')
b24.get_dataframe(res)
Iterate through query results
res = b24.query(symbols = ['AAPL'], as_of = '2020-01-25', back_to = '2020-01-20', name = 'DXOpen SF')
for batch in res:
batch.to_pandas()
List of Datasets
b24.datasets()
Returns a list of all datasets
Caution
Liberator and Rosetta can be used in both CQ Mariner Backtester and CQ AI.
There are no restrictions on the dates and times you can request.
When submitting a query in the CQ Mariner backtesting environment please be careful not to access future data.
ie in backtesting the 'as_of' time should be the current timestamp or earlier (ie less than the current time stamp).
Script from cloudquant.interfaces import Strategy
import liberator
import rosetta
import pandas as pd
pd.set_option('display.max_rows', 500)
pd.set_option('display.max_columns', 500)
pd.set_option('display.width', 1000)
class LiberatorDemo(Strategy):
@classmethod
def on_strategy_start(cls, md, service, account):
dataSet = 'ETFGlobal Constituents'
asof = '2020-05-01'
backto = '2020-05-01'
symbols = ['DIA'] # submit a list or use None for all symbols
df = liberator.get_dataframe(liberator.query(symbols = symbols, as_of = asof, back_to = backto, name = dataSet)) # Query and converstion to dataframe in one line! Nice Python!
df['Date10'] = df.apply(lambda row: str(row.timestamp)[0:10], axis = 1) # Trim time off timestamp as it is midnight 00:00:00
print(df[['Date10', 'weight','constituent_ticker','constituent_name']]) # Display just a few columns
Date10 weight constituent_ticker constituent_name
0 2020-05-01 0.083510 AAPL APPLE INC
1 2020-05-01 0.082193 UNH UNITEDHEALTH GROUP INC
2 2020-05-01 0.063143 HD HOME DEPOT INC
3 2020-05-01 0.052769 MCD MCDONALDS CORP
4 2020-05-01 0.051163 GS GOLDMAN SACHS GROUP INC
5 2020-05-01 0.050721 V VISA INC-CLASS A SHARES
6 2020-05-01 0.050432 MSFT MICROSOFT CORP
7 2020-05-01 0.042929 MMM 3M CO
8 2020-05-01 0.042840 JNJ JOHNSON & JOHNSON
9 2020-05-01 0.038530 BA BOEING CO
10 2020-05-01 0.035511 WMT WAL-MART STORES INC
11 2020-05-01 0.035207 IBM International Business Machines Corporation
12 2020-05-01 0.033748 PG PROCTER & GAMBLE CO/THE
13 2020-05-01 0.032032 CAT CATERPILLAR INC
14 2020-05-01 0.030478 DIS WALT DISNEY CO/THE
15 2020-05-01 0.027748 TRV The Travelers Companies Inc.
16 2020-05-01 0.026939 JPM JPMORGAN CHASE & CO
17 2020-05-01 0.025839 CVX CHEVRON CORP
18 2020-05-01 0.025515 AXP AMERICAN EXPRESS CO
19 2020-05-01 0.024712 NKE NIKE INC -CL B
20 2020-05-01 0.022438 MRK MERCK & CO. INC.
21 2020-05-01 0.017486 RTX RAYTHEON TECHNOLOGIES CORP
22 2020-05-01 0.016603 INTC INTEL CORP
23 2020-05-01 0.016418 VZ VERIZON COMMUNICATIONS INC
24 2020-05-01 0.013173 KO COCA-COLA CO/THE
25 2020-05-01 0.012463 XOM EXXON MOBIL CORP
26 2020-05-01 0.012035 WBA WALGREENS BOOTS ALLIANCE INC
27 2020-05-01 0.011821 CSCO CISCO SYSTEMS INC
28 2020-05-01 0.010874 PFE PFIZER INC
29 2020-05-01 0.009802 DOW DOW INC
30 2020-05-01 0.001288 US DOLLAR
31 2020-05-01 0.000168 UNITED TECHNOLOGIES CORP WI COMMON STOCK